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An Invest 1 gat Ipn Is made into the problem of construct ing 
a model of the appearance to an optical input device of scenes 
consisting of plane-faced geometric solids. The goal is to 
study algorithms, which find the real Straight edges in the 
iccncs , taking into account smooth varlatfons in intensity over 
faces of the solids blurring of edges and noise. A general 
mathematical analysis is made of optimal methods for identify- 
ing the edge lines in figures,, given a raster of intensities 
covering the entire field of view. There fs given in addition 
a suboptimal statistical decision procedure, based on the model* 
for the identification of a line within a narrow band on the 
field of jfew given an array of intensities from within the 
band, A computer program has been written and extensively 

tested wnlcti implements this- procedure and extracts lines 
from real scenes. Other programs were written nhich judge 
the completeness of extracted sets of lines* and propose and 
teat for additional lines which had escaped Initial detection. 
The performance of these programs is discussed in relation to 
the theory dsrived from the model, and with regard to their 
use of global information En detecting and proposing lines. 
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CHAPTER 1 

TOWARD A THEORY OK THE OPTIMAL USE OF INTENSITY 

INFORMATION IN THE DETECTION OF LINES IN A 

VISUAL FIELD 

I. INTRODUCTION AND DEFINITIONS 

We are interested in studying proc edu r eg which identify 
the images of straight lines in some two- dimensional (picture -like) 
projection nf an array of real objects, given some large set of 
intensity values from points on the two-dimensional p rejection 
plane. We ah all confine our attention to arrays o£ plane -faced 
solids. Applying a Line -detection procedure- to the image of an 
array of such objects yields a 1I description"' of the objects which 
is, in effect, a line drawing of the objects from a particular 
point of view. Such a description ja complete in the sense that it 
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contains all information obtainable from that point of view which ia 
relevant to determining the shape, location in space, and bo. forth 
of the objects. Information of the latter sort, which may be 
regarded as constituting a more detailed description of the array 
of objects, may be derived from one or several f, line drawings" 
of a particular array of objects j by methods surveyed, in the last 
chapter of this paper - 

In the first Lhapters of this paper, we shall investigate the 
prublecn of cojls' ructir.g lint detection procedures which make 
optimal use of the set of intensity values to which they are 
applied, by being maximally accurate, in various senses, about 
the lines they claim to exist. Such investigations require a 
formalism, definitions and so forth, which will presently be 
developed in this section- In the meantime we shall informally 
discuss several considerations relevant to the development of 
such a theory, with the idea of motivating some of the formal 
presentation of the rest of the chapter. 

First, it appe-ars thai a theory of optimal edge finding 
procedures would be intractably difficult if we were Lo consider 
procedures which decide on the locations at which to measure 
each successive intensity value- This leads into the realm of 
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sequential decision proceduresj and we shouLd not attempt this 

until we first have a reasonable understanding of the simpler 
non- sequential case, One may get an idea of the inherent 
complexity of optimal sequential decision procedures by 
consulting, e. g. , Bellman (Bellman), For the mometil, then, 
we shall assume that the intensities are obtained from the two- 
dimensional projection of the array of objects in a predetermined 
pattern, such as at the vertices of a grid of squares, and that the 
line finding procedure acts on exactly these intensities. We 
shall return to the problem of sequential analysis of scenes in. 
chapter four, where the matter is approached at the heuristic 
level, 

A second assumption to be made concerning the set of 
intensities upon which a line finding procedure acts concerns 
the relation between the actual intensity at a point in the two 
dimensional view of the scene and the intensity available to 
the line -finding procedure ^ which would be obtained by a real 
optical input device, In general such a device is subject to 
yarioufi forma of optical blurring and distortion, as well as to 
random noise. If there is nothing known about the noise, then 

10 



the &et of intensity values obtained by Such a device contains no 
information about the array ol objects at hand, in particular 
about the locations of the projections of their edge lines * li is 
a re as unable assumption that the obtained values of intensity at 
any point be normally distributed about the actual intensity, and 
that the standard deviation of such a distribution be a function 
nf the intensity at the point. It is further reasonable to assume 
that any soatiai distortion of the intensities be accountable for 
principally as some sort of blurring phenomenon. These forms 
of distortion and noise are present in any real optical or electro - 
optical system; and certain devices, such as an image -dissector, 
may be designed so that these are the principal effects, 

A third restriction involves the relation between the size 
of the objects in the scenes under eonsiderationj or more 
properly the size of the images of these objects in a two 
dimensional projection, and the spatial density of points at which 
intensities are measured. If the intensities are obtained: in a 
simple grid pattern with a spacing of one unit on the image plane, 
it would clearly be impossible to detect edge Lines whose images, 
for example, were shorter than one unit- In judging the 
optirnality of an edge detector, it would surely be unfair to 
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penalize it for npt detecting so short a line. A solution to the 
problem might be to consider only arrays of ohjects whose 
images are larger than a certain size relative to the spacing of 
the points at which intensities are measured. An alternative 
approach, which will actually be employed, is as follows- Chocs e 
a suitable geometry of points in the image space at which to 
measure intensities . Then define a simple predicate on exact 
values of intensity which would be obtained at these points in the 
absence of all noise, distortion and imperfections ir. the objects 
being examined. Then use this criterion far the existence of a 
line to judge the performance of a line detection procedure which 
measures intensities at the same points, but with a real optical 
input device. By this means, one is essentially determining how 
well the procedure ''sees through" the distortion and noise; and does 
not penalize it far not ''seeing" lines that it could not possibly 
detect from intensities measured at points of the given grid 
even in the absence of noise. 

The concepts so far discussed maybe summarized in the 
following definitions. 
Definition; an intensity function , Ifa. yfr, is a real -valued 

function whose values and derivative? arc dotiisd 
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on a square in 2 -space, except for a set of 

measure 0. 
Definition: a description space, D, is an arbitrary set of 

elements {dj|. 
Definition; a sampling set J 3, ia a finite set d£ points 

{iH-y^ -- 1 

Definition: a. noise-free sample . 1(5), of an intensity function 

Ilx, y)j is the set of values of Ife,, y{l for the 

points (x^ y.) of S. 
Definition; a noisy sample J(S), is defiued as above, but with 

the values of intensity modified by some distortion 

and noise functions.. 
Definition: an identification function, F(IJ, is a mapping from 

a set of intensity functions onto some description 

space, 

F: {ij ->D 
Definition r a noise -free sample identification function, GQjW)}, 

is a mapping from a set of noise -free samples onto 

a description space. 

G: (i(S)f ->D 
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Definition: a noisy -sample identification function , HU(S)), , 
i& a mapping from a sot of noisy samples to a 
description space. 
H: JJ<S>) ->D 

Definition: a a ample identification function G, together vvith. a 

space Of nOiSO-frce Samples \l(§}\ and a description 
space D are said to be descriptively complete with 
respect to an identification function F if 

This definition describes the situation where all that one wishes to 
say ah nut a scene may be inferred from the values of intensity at 
a predetermined finite set of locations on the visual field. 
Definition; an intensity function J*Ik, y) nn a domain A is said 
to be equivalent onder the blurring function 
B(^j.y) to a function Ite, y) if: 
I*(x,y)= / I(u, v>H(u-x p v-y)du dir 
A 
Definition: an intensity function I**(x, y) is said to be 

Gaussian -noise mediated with respect to an 
intensity function I*(x> y) if successive values of 
[**(**, yj distribute normally about I*U,y Y q ) with 
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a standard deviation which is dependent solely on 
the value ol I*fc * y fl >. 
Definition: a transducer which goes from an intensity signal 
ItXj y) to a Gaussian -noise modulated function. 
I**Uj y) of a function I*(x, y) which U equivalent 
under blurring to I(x, y), is called a transducer 
subject to fundamental distortion. 
Definition; a scene system with identifiers consists of: 

{L(x, y) |» & set of intensity functions on a common 

dociiainj 

jCx ±a y.) I - S, a sampling set; 

U-(S)r a set of noise-free samples, one for each 

intensity function 1^; 

\ JAS)\ t a set of noisy samples, an infinite number 

oi J IS) for each L(S). The rel&ti&n between a set 

of Jfj(8) and the corresponding l^S) is entirely 

accountable for by fundamental distortion. 

{dj| = D, a description space; 

F: ll-(x,y}| -&D, an identification function; 

Gi 0{tS3} -^D, a nois*-free sample identification 

function; 

H: jJjtS)| -s-D, s noisy -sample identification function, 
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Finally, for completeness; 

Definition; a scene system with identifiers is said to be 

descriptively complete if the relevant parla conform 
to the descriptive completeness criterion above. 
The various restrictions set forth in the earlier parts of 
this section may be simply summarized by saying that we will 
concern ourselves exclusively with Scene systems with identifiers 
which are descriptively complete- 

II. COMPLETE SYSTEMS OF STRAIGHT-LINE SCENES 

A criterion for what constitutes a line in a noise-free 
sample will now be given. This definition will provide a basis 
for a definition of a line in an intensity function which will be 
appropriate for ensuring completeness. 

Prior to making this definition, however, it is necessary 
to digress slightly to consider a plausible restriction on the 
geometry of the sampling set, An important desideratum of any 
line finder is that it not be especially sensitive to lines in a 
particular part of the field of view, or at any particular 
orientation in the field of view. It may fc>e that, of two procedures, 
one will find more lines with a given amount of effort, but at the 
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expense of uniformity iD this sense. One would still not prefer 
the ^better" line finder, on account of its inhomogeneous or 
anisotropis behavior- One thus wishes to guarantee that any line- 
finding procedure that i£ a candidate for being considered optimal 
is in addition uniform in this sense, A necessary condition seems 
to be to have a sampling set which is distributed uniformlu over 
the entire field of view. Without proving this Last assertion, we 
shall henceforth restrict our attention to systems whose sampling 
sets consist of points on all vertices of 6ome grid of squares 
CC-vering the entire field of view. 

A Straight line in a nniSe-free sample over a sampling 
region as described above may be defined in terms of the notions 
of discontinuity and anomaly. We define three Such notions 
for, e, g- , a set of eoLinear intensities q, i^j ij and i^ using 
constants K, K 1 and K' 1 : 

1) A discontinuity of intensit y occurs at a point between the 
points at which ir, and 1 were obtained if: 

|i 2 - i-| > K(|i^ - i 2 | + |i g ■ i 4 l ). 

2) An anorn aloua intensit y occurs at the point where ig 
was obtained if: 

|ig - (ij - 1^/2 [ > K'. 

L7 



31 A discontinuity in shading occurs at a point between 

the points &t which i and i were obtained if- 

^ 3 

iUi-y-u 3 -Vi>K". 

A line may be defined as two or more such points, all of the 
same type and all adjacent and colinear. 

It ie now easy tc- specify a class of systems with identifiers 
which are descriptively complete in their identification of linos in 
a scene. One may simply choose a scene system with a description 
space appropriate for describing lines, and define a noise -free 
sample identification function G in such a way that it appropriately 
identifies all the lines in a noise -free sample according to the above 
definition of a line. One may then define the identification function 
F as follows: 

FCI}^*^ Ga(S)J=d i - 
The completeness follows a fortiori. 

In whatever system we shall henceforth be considering, we 
shall assume that the description space is the power set of the set 
of all lines definable as above on noise-free samples. We shall 
further assume that the noise=free sample identification function 
of the system is as defined in the previous paragraph. The 
choice of an optimal noisy- sample identification function for this 

type of system is the subject of the remainder of this paper. 
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III. OPTIMIZATION CRITERIA 

The variables to be considered In discussing optimal 
object recognition algorithms are. number of intensities read, 
time taken to read each intensity value j length of computation 
time, and rate of error in identifying lines.. We will not be 
concerned with, computation time, as this would require more 
knowledge about '"mimiTnal" versions of the algorithms concerned 
than i.a available today. On the other hand, the number of points 
at which intensities are taken, and the length of time taken in 
getting the intensities can be related in a natural way to the 
accuracy of identifying and locating lines, or other features 
of a scene. Intuitively, there must exist a trade -off between 
the measurement uu£t of a certain amount of intensity 
inform at ion; and accuracy, measured in terms of number 
of lines correctly identified. Optimization may consist in 
minimizing some type of error rate-., with a fixed overhead 
in effort. One may., alternatively, wish to set a certain 
error rate criterion, and nunimize the amount of effort 
necessary to achieve it. It is easy to see that this latter 
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form of optimization may be reduced to the former' Suppose there 
exists a procedure F(E) which, using E units pf effort, minimizes 
the error rate. If it is desired to minimize effort for a fixed 
error rate,, one may simply consider all possible instances of 
the procedure P(E) with error rate below the desired one, and 
choose the procedure P(E+) where E* is a minimum among all 
such amounts of effort. This procedure u£es the least possible 
amount of effort while guaranteeing the error rate to he below a 
certain value. We may thus without loss of generality restrict our 
attention to procedures which make a minimum of errors for a 
given amount of overhead* 

Further discussion of error rate requires a more detailed 
discussion of various types of classification errors and the 
probabilities with which they occur. We shall base this discussion 
on the nation of false positive error, the assertion by aline finder 
that a line exists at a particular place on the visual field when no 
such line exists in the scene under consideration, and upon the 
converse nation of true negative error. We will begin with some 
definitions regarding line identification functions. 
Definition; In a system of the type under consideration, for 
some noise-free sample 1.(5) and a corresponding 
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Definition; 



Definition; 



noisy pattern .L(£>, we call the set of lines 
H(Jj(S)>-CtfLlS}) the set of false positive errors 

in the identification of the Scene from the noisy 
sample J ASK 

For similar conditions, xhe set GU^SD-HtfJS)) 
is called the set of true negative errors in the 
identification of the scene from the noisy sample 

J-jCS). 

Let A And B be two sets p then the difference 
cardinality, N(A,BL is the cardinality of the 
set A-B+ 
It follows immediately that for any J (S) and corresponding 
Ij(S) t the number of false positive errcira is given by: 

wmu.tmansm, 
■I 

and' the number of true negative errors by: 

N(G<I i {S»,H(J. j {Sm. 

With the aid of the function N defined above, we may 
indicate the performance of a noisy sample identification function 
in terms of the number of errors of each type it will make in 
describing any particular noisy sample. Before being able to 
indicate the over -all performance of a noisy sample identification 
function, one must as well be able to specify the relative 
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probabilities of occurrence of the various pairs of noise -free end 
noisy samples. For the present we need only the first of the 
following definitions; however they will all ultimately be necessary. 
Definition : PC^ ( S ), J } ( S) ) is th e joint pr o b ab ility of oc car r ence 
of the noise-free sample I^S) and the noisy sample 

Definition; P(Jj(S)/l i (S)) is the conditional probability that the 

noisy sample J^S) will occur given the occurrence 

of noise -free sample IdS)* 
Definition; Ptysj/jjtS)) is defined similarly. 
Definition: FUtfSM, P^CSI) are the a priori probabilities of 

occurrence of the noise -free sample i^sj- and the 

noisy sample J-(S), 
We may now discuss the expected rate of error of a 
classification system of the type under consideration. 
Lemma: The false -positive error rate^, or expected number 

of false positive errors made by a complete scene 

system is given by: 

2><MS), J,(S))NfH(J.(S)K G<L(5))> 



i-J 



Proof; 



Whenever the noisy sample J.fS) occurs in 
conjunction with the scene re presentable by 
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the noise -free sample !•(&)> the number of 
false positive errors is: 
NtHtJJS)}, CKUS))). 

This occurs with probability 

PU.lShJ (S3). QED 

1 3 

Lemma; The true negative error rate in a similar situation 

is given by; 
X PiysL J.(S))N(CKl t (S)L H(J-j(sm, 

the proof is similar. 
With the aid of the following definitions, we may define two 
additional err or -rate measures; 
Definition: M(A, Bh tp.e total number of errors is defined by: 

M(A I B)=N(A,B>tN<B < A). 
Definition; G<A,B), the identity predicate , is defined by: 

0(A,B)=1 ifA^B, 

CXA,B)=0 ifA#B. 
Lemma: The correc t recognition rate., or expected rate at 

whic:h a scent is perfectly identified i& given by 

the following: 



T F(I. (S), J .[S»CK(3(J IS», H(L4S})> 



the proof ie as above. 
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Theorem; The total error rati-, or expected number of errors 
made by a complete scene system is given by: 

the proof is as above. 

As a corrolary to the above theorems, we make the 
following definitions. 

Definition: atf^S), d.)= ^P^Sy/j^Md^ Ctt^S)}) 

is the expected rate of false positive errors on 
occasions when the noisy sample J^S-J occurs, 
provided the description d. is assigned to it. 

Definition; btf^S), d^^FO^SW Jj<5»N(G(l.(5», d^ 

is the expected error rate of true negative errors 
under similar conditions. 

IV. OPTIMIZATION PROCEDURES 

With the aid of the definitions in. the previous section, we 
axe in a position to discuss four optimization procedures. The 
first involves the maximization of the probability of cp-rrectly 
recognizing a scene in its entirety. The second involves 
minimizing the sum of the false positive and true negative error 
rates. The last two involve the maximization of ont of the error 
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rates with the other error rate held constant. Since these latter 
two procedures are mathematically nearly equivalent, we shall 
consider only the c&ee of holding the false positive error rate 
constant, and minimizing the true negative error rate. 
Theorem: The scene recognition procedure which maximizes 
the rate of correct recognition a£ Scenes in their 
entirety is given by the function H{Ji)-d, such that 



Proof: 



is maximised. 

If description d^ is assigned to the noisy sample 
J|(S)j then a perfect recognition of the scene will 
occur whenever L{S), the corresponding state of the 
real world, has the property that GU^S)^^. This 
occurs with a probability P(I.(SJ/J (S)) for a particular 
I AS) having this property. Hence the expected rate 
of correct identification for the noisy sample 



J. (5) is: 

i 



This is equivalent to; 

XCKGd^S)), d k )p(l t <S>/J,(S)) 
i J 



or: 



/PUjtSJ) TcKGKlitSH, d k )P(I i (E), Jj(SJ). 



as 



Theorem: 



For a given J (S), the rate of correct recognition 

is obtained by maximizing- this sum. This 
aplijjr:is::-tioj; tor a p F.r *icui Ar J.l.S) it independent in 
its effect on optimizing the recognition of other 
JjJS). Hence this noisy -a ample -wis* optimization 
procedure provides an optimum recognition procedure 
for the set of all J.(S) + 

The scene recognition procedure which minimizes 
the rate of false positive errors plug true negative 



errors is given by the function HtJ.(S^)^d t with the 



■'k 



Proof; 



property that: 

3£ M( G{ I A ( 5) ), 4^ )p< y S) , J|( S) ) 

i " J 

ifi minimised. 

Similar to that of the preceding theorem. 

The last case is a bit more complex* 

Theorem; Suppose it is dee ire d to keep the false positive error 
rate at or below a constant value a, while minimizing 
the true negative error rate. Then there exists a 
c, depending only on a. with the property that the 
desired optimality may be achieved by assigning to 
J. IS) the description d such that 

J -K 

2 6 



oafish d k ) + {L-cJb(J j (S) J d k > 

is minimal for all d^. 
Proof j 1) There must exist a description function having 

false positive error rale at or below or,, and minimal 
true negative error rate. This follows from the 
fii'Lit :bat, e.g. , H(J;(S)> = a gives a false positive 
error rate of 0, Hence the set &£ decision functions 
with, false positive error rates below a is non- empty. 
There niml i?:;:l.I :>ill; with minimal r r r\j& negative 
error rate. Let tt be denoted by mj.tS)}, and its 
true negative error rate by J& 
2) Consider a decision Junction H*(J,.{S)) which 
assigns to Ji(S) the description d^ such that 
x-atJjtEH, d k ) + (1 -x) b(j.(.Sh \ ) is minimal for all 
d^. We shall show that for a suitable value c of x r 
the function H*(J-(S))' has a false positive error rate 
at or below a. and a true negative error rate of £, 
From this it will follow that the function H*(J,(S)h 
which is the funetic-n referred to in the theorem, has 
the -desired optimality. 
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3) Let: 

b^bfJ^Sh HU^SW; and also 

bjg(j)-ttJj(S},H£(Jj{&>». 

From the definition of h*(Jj(S)), we know that for 

allj, 

x-a(j> + (l~x)b(j) > x-ajtj) r (l-x}b*(j). 

Multiplying by P(Jj(S)) and aumming on j, we have; 

.1 J 

x^^]JP(Jj(S)J + U-3t)5^b*(j>PtJ,<S)). 

Butj by definition., 
Ya(j)P(J.(S)) = * t and 

7 J 

J 

Also, letting 

2 a *{J> p U|<S}) = a*, and 

.1 

j 

we have: 



2e 



4) It is not difficult to see that x may be chosen 50 
as to make ** = *. This follows by assuming that 



a* is a continuous function ol ^; and noting that 
* = 5, since H* minimizes false positive errors 






ignoring true negative errors, and HUrtS?) = 
achieves & false positive error rate of p T By similar 
reasoning, «^ increases without bound as x. decreases 
from 1. Thus for some value e of x the value of 
a * =a r This approach maybe applied to the 
discrete ease, ■where at least a* will be very close- 
to a. The expression of the previous step will then 
reduce to: 

ca - U-c)0 > ca + (l-c)0*. 
But is minimal, and U -c> > Q, thus: 

&= 0*. Qh-:D 
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CHAPTER 2 
THE OPTIMAL USE OF REGIONAL INTENSITY INFORMATION 

IN THE DETECTION OF LINES 

I. THE REGIONAL QUALITY OF ISOLATED LINES 

Id this section we shall direct our attention to situations 
where there exist no systematic spatial relationship* among the 
lines appearing In scenes under consideration. We shall briefly 
discuss the extent to which lines in this case are "regional" in the 
aense that all intensity information relevant to the detection of a 
line is contained on the visual field within a certain short distance 
from the line. 

Definition: Let S be a sampling region consisting of points 
uniformly distributed over a field of view. Let 
F<A,B, S) be a predicate on intensity values at points 
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of S, which attempts to decide whether or not there 
is £ line segment through points A and B. Let a ' 
and respectively be the faLse positive and true 
negative error rates of the function Ft A, B, f£)> 
Then the quantity & F (*) is defined as the smallest 
distance such that there exists a. predicate F*(A, B) 
which decides on the existence of a line through A 
and B only on the basis of intensities of S lying 
within this distance of AB, and which has false 
positive ana -.::\'-: negative error :li'.l-j respectively 
below fl+* / and £+* . 
For seme small value of *, the value of £_<*) may take c-n a 
wide variety of values in different sorls of situations- For example 
6 _.(* ) would he large in the situation where scenes consist at 

if 

rectangular parallelepipeds. In this case, Lines parallel to a 

proposed line, or incident with an end, constitute useful evidence 

for the existence of the proposed line. The spatial remoteness of 

useful evidence in this case might make Sp,(*) not appreciably 

smaller than the radius of the field of view for small values of * 

However, S_(i) may be relatively small in the case of scenes 
F 

consisting of single discontinuities of intensity across the field, 
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or of sets or straight lines drawn at random across a sheet of paper. 
A hypothetical situation of particular inter ^t is that in which scenes 
consist Of lines which, within a relatively large radius, behave like 
lines given rise to by edges of plane-faced Solids; but which are 
randomly distributed across the field of view. In eases such as 
this, two effects limit the predictive value of intensity information 
remote from a proposed line. 

The first effect has to do with the unpredictibility of 
intensity values away from an edge. In the absence of optical 
distortion and noise, intensities on a line normal to an edge might 
appear as In Figure 2.1, The determination of the existence of an 



HYPOTHETICAL IDEALIZED 

INTENSITY PROFILE AT 
AN J£]>GE (X n ) 



INTENSITY 



X 



Figure 2. 1 
Intensity discontinuity at Xq Is dependent upon finding the right and 
left limits of the intensity function at the point x„ The intensities 
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along the normal away from x contain information about the left 
and right limits al i . However the amount of information contained 
in these intensities falls off as one goes farther from the edge. This 
is due to possible unpredictable ncn- uniformities in the illumination 
and reflectivity o£ the objects in the scene. 

The second effect has to do with the nature of the blurring 
inherent in an optic al system. Aji edge subject to such blurring 
appears as in Figure 2. 2. An abrupt step function is smeared out 

INTENSITY PROFILE AT AN 
EDGE (x ) SUBJECT TO 
BLURRING 



INTENSITY 



*0 
Figure 2.2 

a small distance from the edge and causes extreme values of the 

various derivatives e.1 the intensity function in a small neighborhood 

of the edge. R is the extreme values of the derivatives that 

characterize and identify an edge> However, the effect of the 

existence of an edge on the derivatives falls off drastically as one 

goes away from the edge. Hence, derivative information on a set 

of intensities remote from a possible edge can give little 
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information about the presence or absence of the edge. 

The spatial extent of this effect is determined by the 
critical resolution radius defined as follows. 
Definition: The critical resolution radius is the radius of a 
circle whose intensity equals (I+J>/2 around the 
image of a point of intensity I on a background of 
intensity J, 
This radius is related to such factors as the resolving power nf 
the optics of the system. Empirical evidence suggests that for 
relatively small values of t t the value pf d^t) is on the order of 
magnitude of a few times this radius, in the hypothetical case 
mentioned of randomly distributed par allele piped -edge -like lines. 

In situations where the existence of a line is indeed 
virtually determinable entirely from intensity information contained 
within a few critical radii of the proposed line, we can test £nr the 
presence of a line in a given location by examining an area which 
could only possibly contain a few other lines resolvable from the 
given line. In general there would seldom be any line but the 
central line in an examined region. This suggests the development 
of a theory of the optimal use of intensity information within a 
narrow rectangular region, based on a model which assumes the 
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possible existence of at most one line in the region, 

II. A REGIONAL VERSION OF THE GENERAL THEORY 

A regional version of the theory presented in the previous 
chapter will now be discussed. The theorems presented give rise 
to & procedure which is optimal in deciding on the presence or 
absence of lines within a narrow rectangular region -, 

For the theory to be discussed, we adapt the following 
from chapter 1: 

Let { I (?t, y) \ be a set of real valued functions defined 
QVST a narrow rectangle £L 

Let D = f 0, ] \ be a two element description space. 

Let S, a sampling set, be a finite subset of R, 

Let F{I-(R)lbe an identification function mapping onto D, 
with the inverse image of the element 1_ consisting of exactly the 
intensity functions given rise to by lines down the center of R. 

Let GU|(S)) be a noise free sample identification function 
such that it and the function FU^R)) conform to the descriptive 
completeness criterion. In this case, descriptive completeness 
requires thai any line identified on R by F(I t (R» be identifiable 
by GCI|<S» from intensities on S. 
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We shall make the following assumptions as well; 

1) That the intensity gathering device is subject to 

fundamental distortion, as defined in the previous chapter. 

In particular, we assume thai this distortion governs the 

relationship between noisy samples J-tS), and corresponding 

noise free samples IJS}. 

i 

2) That at most one line can occur in the set S* 

Some of these definitions and assumptions will not be explicitly 
used until the next chapter . In particular, the theorem abnut 
to be proved makes rather little explicit use of them* 

The optimization of the system in question will, as before, 
be a matter of making optimal ust of a series of a priori and 
conditional probabilities in the minimization of certain error rates. 
In particular, we will assume that the a priori probabilities 
Pd^S) of each noise -free sample MS), together with the 
conditional probabilities P(J|{S)/ 1.(5)1 are known; or alternatively 
that the joint probabilities P(J.{S), 1.(5)}, the product of the previous 
two* are known. Again we may wish to hold the true negative 
error rate to some value and minimize the false positive error rale; 
hold the false positive error rate to a certain value and minimise 
the true negative error rate; or minimize the sum of the two. 
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A surprising result emerges from IJtl i theorem about to be discus &ed: 
that these three types of optimisation are brojght about by 
essentially the same procedure, 

We need the following definition before stating the main 
theorem of this section. 



Definition: 






(2. 1) 



This quantity is the prob ability , given the noisy sample J(S)., that 

there is a line down the center of the region R. 

Theorem; Suppose it is desired to keep the false positive error 
rate below a certain value a, and to minimise the true 
negative error rate. Then there exists a £ such that 
the following decision function E(J(S)) satisfies the 
conditions: 

H(J<S))=l»Q<JlS))>fl (3,2) 

Where $ is the minimum value such that? 

Q(J|(S))>0 



I 1 :■■■■/: 



1) Suppose a noisy pattern J(S) occurs, and that 
H(J(S))=1, i. e. a iine is claimed to exist. Then 
(1-Q(J<S))) is the relative probability of a false 
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positive error whenever this pattern J{S) occurs, and 

(L-Q{J(S))PLJ(5)]i is the absolute probability of a false 

positive error involving the pattern JT(S)h Thus? 

J^i-WiOHFms)) 
H(Ji(S))=l 

Is the over -all false positive error rate of the function 

H{J[S», 

2} It is clear that the value l_ in place o£ ^ in 

equation (2. 3) satisfies it, and that such values are 

bounded below by 0. Hence there exists a least value 

satisfying the Equation, which we shall call £. 

Also, in the continuous cp.se, for this value of the 

equality of equation (2. 3) will hold. One may 

consider this to be the case as well in the discrete 

case, if the number of discrete values is large. 

Z) It remains to show that this procedure gives a 

minimum number of true negative errors. This error 

rate is given by: 

^QQjISDP^S)) (2,4) 

H(Ji(5))=0 

Suppose we consider another decision function 

H*(J(S)) satisfying the condition of having faJse positive 
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error rate equal to or less than a , i, e, 

^a-Q(J £ (SHP(Jj<S»<fl (2.5) 

H*U i (S))=l 

We wish to show that its true negative error rate 
i,s not less than thet of rHJ(SjJ* given by expression 
(2. 4}, i.e. that 
^QU^SMFU^S)) > J^OWifSHPtJ^S)) (3,6) 

4) Let R be the set of i's s*t r fftJ^SH-l, and 

and let T he the set of i's &,t. H{J.(S))*0, and 

and let U be the set of i's s. t, HU i (S)^*0, and 

H*{J.(S»=0, 
and let V be the set of i's e.t. H{J^E))=l, and 



H^J^S))-!. 



Then 



if 1 H 
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and 

So to prnve {2, 6)> we need to show; 

ybu f (£))FUJS)} > JfilJ,(S»P(J f (S)) (a. 7} 

R T ' 

IX UR, thcnQ<J|tS)} > 0,and if ifT, QiJ^S)) £ 0, so 

^QtJ^SJjPtJ^S)) > 0^ PCJ^SH {2. 3> 

K R 

and 

^QtytSttPU^S)) < (8^ P(Ji<SH. (2. 9} 

T T 

Assuming equality In expression (3, 3), we have 

from (2. 3 J, (2. 5 J and the definition of R, T, and V; 

J/l-Qt.USHPU.tSH ^ > Jl-QIJ^SJlHUfSlI, 
RUV TUV 

or, expanding and eliminating the sum on V: 

2><ji(S)> - ^t?{j (s)>f>i: j £ {S)) > 

E R 

T T 
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Combining- (2. a J, (a. 9) and (2.10): 
R R 

R. R 

^PU^S)) - ^QUjtS^PQ^SDi 

T T 

T T 

or: 

(i/3-i) 2j5(Ji(S))P(Ji(s» > (i/3-n^Q(j i (.s))p(j 1 (s)). 

R 

Formula 12. 7) follows from the fact thai (1/0-1) is 
positive. QED 

The ease of minimizing the false positive error rate with 
a fixed true negative error rate is almost Identical; 
Theorem: Suppose it is desired to keep the true negative error 
rate below a certain value a, and to minimise the 
ialse positive error rate. Then there exists aj3 
such that the following decision function satisfies 
the c.ji\::: I i ;>:-:-: : 
H(J(S))-1<E=> Q(J(S))i0 . 
Proof: As for the previous theorem. 

It follows from these two theorems that whatever the value 
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r>f ^ used, the procedure of thresholding Q(.J.(S)) is better than 
all other procedures with, the same false positive error rate h 
and belter than sdl other procedures with the same true negative 
error rate r In this respect it is similar to a saddle point solution 
of classical 2 by 2 game theory. 

It is not difficult to see that this procedure can, for * 
suitable choice of fi, be used to minimize the Sum of the two 
error rates. Consider an arbitrary choice procedure H* which 
gives a false positive error rate a and true negative error 
rate b. Then there exists a j3 such that thresholding Q(J{S)) at 
jS gives false positive error rate a- and true negative error rate 
b* t where b* ^ b. This will provide a decision procedure 
whose total error rate is a +■ b* < a + b* Hence thresholding 
of Q(J(S)) with this £ gives a minimal suru of error rates. 



IIL THE RELATIONSHIP BETWEEN GLOBAL AMD 

REGIONAL LINE PREDICATES 



This, section discusses the relation between the global 
line finding procedure described in chapter one, and the 
regional procedure of the present chapter,. It is clear that the 
procedure of chapter one Ls computationally unfeasible- 
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However a systematic application of the regional predicate- '.u a 
large number of regions on the field of view may be computationally 
practical. The latter prcic.edu re, in certain cases of interest* can 
not achieve the optimality of the global procedure. However it 
does trade a certain amount of accuracy for computational 
feasibility. 

It has been pointed out that in certain cases where ILties 
on the visual field bear no systematic relationship to each other,, 
a regional line predicate of the sort described in the definition 
at the beginning of this chapter Seems to be potentially as 
accurate as a global one. It is tempting to formalize this notion 
into some sort of theorem,, stating conditions under which a 
systematic application of a regional predicate of this sort to a 
suitably large number of regions on the field is equivalent to a 
global procedure. Unfortunately, an exact correspondence 
between these two procedures is difficult to make. One problem 
is illustrated in Figure 2, 3. It is reasonable to assume that in 
the system described in chapter one* if an element of the 
description space contains a segment AB, it would not also 
contain a subsegment of that line. More informally stated* the 
scene descriptions would not contain redundant segments, 
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A REGIONAL LINE PREDICATE 
APPLIED TO MANY REGIONS ON 
THE FIELD WOULD REPORT LINES 
Afi 3 AC, . . . , AZ ± BC, . >. , BZ J ETC. 
A QLQBAL PROCEDURE WOULD 
OMIT THE SUB -SEGMENTS AND 
REPORT ONLY AZ. 



Figure 2.3 
On the cither hand, the systematic application of a regional 
predicate of the sort mentioned would not only give positive 
identification for some complete line in the scene, hgt for all 
examined subsegments as well. Another difficulty is that the 
regional predicate might report the existence of two nearly 
coincident lines at very slightly different inclinations, wh#r* 
there is actually only one line. These difficulties will he 
encountered in chapter four, which describes an actual computer 
program, embodying a regional line predicate. In the program h 
the problems are handled at the he ur is lie level. How ever , from 
a formal point of view, obvious efforts to solve them appear 
artificial. We will thus confine the remainder of this discussion 
to the informal level. 
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It is convenient to informally divide the evidence relating 
to the existence of a particular line into three categories: 

1) Global evidence j namely evidence from areas spatially 
remote from a possible line; 

2) Regional evidence h namely evidence from an area 
within a small radius of a possible line; 

3) Local evidence, namely evidence contained within a 
neighborhood of some point on a putrsible line. 

It has been pointed out that there are certain problems 
associated with a formal identification of a systematic use of a 
regional predicate with a global predicate. This identification 
appeared plausible only in cases: where there is no relationship 
among the lines In the figures* However, the situation of 
particular interest is that in which the scenes consist of sets of 
plane -faced solids, tn this case there is definitely a relationship 
among the lines in the figures; and a regional predicate cannot 
possibly achieve the optimal ity of a global one. Figures 2, 4 
and 2. 5* illustrate the possible failures- of a regional line 
predicate, which might be avoided by the use of global evidence 
as well as regional evidence. 
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Figure 2, 4 illustrates three lines radiating from a common 



DISTINCT LINES 




LIME IS 
VERY FAINT 



LINE IS FAINT 



THE REGIONAL PREDICATE APPLIED 
TO "A" INDICATES A LINE DOES NOT 
TRAVERSE THE WHOLE REGIOKj 
APPLIED TO "B" IT IS POSITIVE, 
B THIS IGNORES THE GLOBAL 
^ EVIDENCE PROVIDED BY THE 

"DISTINCT" MNKS. WHICH SUGGEST 
THAT THE LINE COM TENUIS TO THE 
VERTEX, 



Figure 2. 4 
vertex. The upper two lines are quite distinct, whereas the Lower 
line is faint, especially at the top. A regional predicate applied 
to region A may claim that a line does not traverse the entire 
region. However., when applied to B, there is sufficient evidence 
to claim the existence of a line. The fact that the line in region B 
"points to 1 ' a close vertex is ignored by the regional predicate. 
Global evidence indicates that the line continues to the vertex. 
The latter would likely be the conclusion of a global procedure, 
as discussed in chapter one, which would assign lew plausibility 
(a priori probability) to Scenes with isolated line ends, and high 
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plausibility to closed line figures, figure 2. 5 illustrates a 




<*A 



THE REGIONAL PREDICATE 
APPLIED TO "A" MIGHT CLAIM 
THAT THERE IS NO LINE. 

mo-.v :•:■.- :■■:;*, gi.ouai. ::vtiji;ni..:i: 
STRONGLY SUGGESTS THERE 
IS. 



Figure 2, 5 
situation similar to that in Figure 2. 4, In this case a line is 
totally ignored by the regional predicate, despite considerable 
global evidence for its existence. 

In chapter four, a program will he described which uses 
a. regional predicate in the analysis of scenes consisting of 
prismatic solids. It is convenient to here describe briefly two 
features of this program, as the current context justifies their 
use. Specifically , the two features are designed to overcome 
the non-glohality of the regional predicate as illustrated hy 
the previous two examples. The sort of problem illustrated In 
Figure 2, 4 i= overcome by the use of what will be termed links, 
which tentatively extend a line to a possible vertex. The sort of 
problem illustrated in Figure 2. 5 is handled by the procedure of 
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proposing lines. The procedure in effect assigns a high a priori 
probability to lines suggested by lines already identified. This 
tends to augment regional evidence for the proposed line. 

A final fact should be pointed out in this context concerning- 
the relationship between, the use of purely local evidence and the 
use of a regional predicate. This is analogous to the relationship 
between the systematic use of a regional line finder and the use 
of a global one. In the latter case, it ia necessary that the lines 
in the scenes be spatially independent in order that the two 
procedures he equally effective. In order for purely local decisions 
as to the existence of aline to be as effective as regional decisions, 
it would, by analogy., be necessary that the points along a particular 
line be spatially independent. This Is* of course, totally contrary 
to the nature of a line. It is thus quite impossible that a 
systematic application of a simple local decision procedure to a 
suitably large number of ncighburbQuds enverjng a region could 
be as sensitive in detecting 1 lines as a regional predicate. By a 
"local decision procedure" is meant a procedure which decides 
entirely on the basis of intensity information from within a 
neighborhood of a point, whether or not a line traverses the 
neighborhood. The output of such a procedure would be a set of 
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points on the visual field, which would be incorporated into lines 
by finding maximal sets of adjacent eolinear points. 

This two stage procedure is a special case o£ a. set of 
procedures which process intensities in a neighborhood, and then 
process the results for a sel of neighborhoods contained within a 
region* Although the particular two stage procedure mentioned 
in ".he previous paragraph can never be as sensitive as an optimal 
regional line finder > there exist two stage procedures of this sort 
which are. This is not surprising, since all regional predicates 
fall into this class, if one regards the first stage to he the 
identity function, and the second stage to be the regional predicate 
itself. There also exist ncn -trivial examples of members of this 
class which are as sensitive as an optimal regional procedure. 
The procedure described in the nesct two chapters involves such 
a two stage predicate. For the first stage, four values are 
obtained frum the intensities in a neighborhood, Sets of these 
four-tupLes, obtained from all the neighborhoods within a region, 
are processed by the second stage; and a decision is made as to 
the existence of a Line in the region. The procedure may be 
regarded intuitively as extracting all necessary local evidence in 
the first stage, and employing all necessary regional evidence 
in the second stage. 
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CHAPTER 3 
COMPUTATIONAL APPROACHES TO THE REGIONAL THEORY 

I. INTRODUCTION 

The object of this chapter Ls to give a computationally 
feasible method for computing values at the function given by 
Formula 2. l of the last chapter: 

CH^srf-l 

It will be recalled that this ia & function of the finite set J(S) 
of intensities obtained by a particular sort of optical input 
device from within some region on the field of view. The 
thresholding of this value provides an optimal regional line 
detection procedure. 

From Bay^e f rule it follows that 
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so that 






QU(SJ) = 

P(J{S)J 



Using the relation 



i::ii.l 






FUfSU^S)) = P(J(S)/l i <&))P(I L {S)) J 



we have; 

Q(JtSH = 



^PtJtSH/l^SDPtySJ) 



GU^SIM 



Sections II and III of this chapter will be concerned with 

the calculation of the values of FUfSJ/l^S)). Sections IV and V 

propose a model for the relative probabilities of the various 

-ic;i^: ;Y-::i- v^L U.:i:L : i^.^). Oa Lul- biiSii of the ■-■tlues of 

P(I±(SHi given by the model, it is possible to express the sums 

in the above expression as integrals,, and lo give a relatively 

simple explicit formula for Q(J{S)]U In the last section uf this 

chapter, this formula is applied to some simple cases of 

image -dissector output. 
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IL MOISE-FREE PATTERNS AND BLURRING DISTORTION 

The value of the conditional probability PUj/i.J Is a 
function of the relation between the two sets of intensities 
J j and I.. Thin relationship can be decomposed into; 

1) a determinate relation between the noise-free 

sample I. and a distorted but noise-free version I*, and 
J J 

2) a simple probabilistic relation between this I* and 

J 

the noisy pattern iL, 
The former will he discussed in this section; the letter in the 
next section. 

According to the definition of fundamental distortion, 
the relation between a noise -free pattern L and a. distorted 
version I* may be expressed in terms of a convolution of the 
former with some function f(x, y). Possible examples of 
f(x,y) are given in Figure 3. 1. Theoretically, from the way an 





HILL FUNCTION PILLBOX FUNCTION 

Figure 3, 1 
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In-ege-diHHec.tor is designed, this function should be cylindrically 

nhaptd ai in the right hand illustration of Figure 3.1. In faeti 
it is more like a gauss I an "hill" as shown in the left hand 
illustration. An intensity function l{x, y) is acted upon by this 
convolution function f (x, y) to produce a blurred intensity 
function I*(jtj y) as follows: 



i r. 



I*(x,y) = / / f(u p vH(x-U,y-v)du dv. 



If we had an edge which is reasonably uniform for a 

relatively long disLante compared with the ''width" of the 
■•.ui.Y'jlutiorj function, we ,--m deHcrit-R the ncmvoluticn affect, eft 
least locally, by means uf a single integral instead nf a double 
integral. In particular, suppose for some local region the 
intensity function i(x, y) represents a line parallel to the y-axis. 
Then locally we may write Kx, y) = I(xJ and the corresponding 
blurred but noise -free intensity function I*{x ( yJ may be 
expressed in this region as: 



I*Cx.y) = / /f(u 3 v)I(x-u)du dv 






-cn-go 



- I I(x-u) /f(u, v)dv 



* 



du. 



S3 



Since the inner integral is a function only of u* we may 
indicate it with a single function name; 



j. 



g(u) = / f(Uj v>dv > 



--ri 



and we have at a reasonably extensive and uniform edge 
parallel to the y-axis; 

■j. 
I*(x. y) = I*(x) = / E(x-u}g(u)du. {3.1) 



■x- 



We may obtain g{q) empirically by scanning at right angles 
across a. very narrow line in the visual field. Let the intensity 
along a line normal to a vertical line in the visual field be 
given by: 

I(x) - b if x < -a, x > a 

Rx} =c -a < x < a. 

Then the distorted version pf the intensity locally around the 
line is given by: 

=■- -x+a 



I*[x> = /l(x-u)g(u)du ■ c/g(u)du + b. 



■ TC 



-:: Li 
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But Litn 



-x+a 



c /gtu) 



= c-a-gi-x), so 



I*U} 3 c<&-g{-x) -i- 4, 

for very n&rrqw Lines, 

Figure 3.2 shows plots of intensity taken along normals 



INTENSITY PROFILES 
(NEGATED) 

NORMALIZED TO HAVE 
SIMILAR AMPLITUDES 



m SCAN 

i.im:i, 




1 UNIT = 1/2 nng VIPI HELD 



Figure 3, 2; 
to variuu.fi vertical lines* Accurdiug to the preceding 
paragraph, these curves should be of the form a-g(x) + b. 



They art shown superimposed and normalized to have similar 
amplitudes. The Lines are all quite narrow relative to the 
width of "the hill plotted, and represent lines of a wide variety 
of intensities. The values are all proportional to absolute 
intensity, having been suitably transformed from vidis sector 
output which i^ logarithmic:. The degree of variation of 
X\ti -normalized amplitude was greater than ten to one; and may 
be observed from the relative noisiness of ;he curves, AIL 
were recorded at a fixed noise level, and the amount of local 
fluctuation of a given line in the illustration is a measure- at 
the degree to which the hill in question was enlarged in the 
normalization process, It should be noted that the shape of 
the curves is remarkably consistent. This indicates a good 
agreement between the theory just discussed and the actual 
behavior of the blurring process in the vldissector. Figure 3. 3 
illustrates a curve obtained from the data in Figure 3, 2 by 
hand fitting. Again, this curve should be of the form 
a.g(xl - b, where g{x] is the one -dimensional blurring function. 
Also illustrated is a curve of the integral of this function, 
obtained by hand integration. Tbia latter curve will be 
compared With a Similar turvc obtained by two different methods 
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HAND FITTED CURVE 
OBTAINED FROM 
CURVES EN FIGURE 3. 2 




5 10 

1 UNIT B 1/2000 VIDISSECTOR FIELD 




1 r 



INTEGRAL FROM TO X 
OF THE ABOVE CURVE, 
OBTAINED BY HAND 
INTEGRATION 



T 



s in 

1 UNIT ~ L/2000 VIDISSECTQR FIELD 
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APPROXIMATIONS TO gfc) AND 
ITS INTEGRAL MAY BE 0BTAI>JE1> 
EMPIRICALLY FHOM THE DATA 
IN FIGURE 3.2 



Figure Zr 3 
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which will now be described. 

Another approach to finding gU) is based on the fact 
that the ptak value of an intensity scan normal to a line of 
non -negligible width a is related in a simple way to thy integral 

a/2 

-a/2 
namely > if the actual intensity along a normal to a narrow 
vertical line is given by; 

HkJ = c if x <: b, x>b + a 

I(x} = c -i- d b < x < b i a, 

tin an I*(3f), the distorted intently function is given by: 

T*{x) = / I(v-i])g{L})du = / (c + IjfcmHgtuHu, 



where I (x) - Q except in the interval tb^b+a), where 
I (x) = d. One may write the latter integral as: 



Ob tt 



OB fib 

; / g(ujdu + / I (x-u)g<u)du. 



But the former integral is a constant, and the latter integraJ 

s-a 



l/g^ujdu, 



is equal to: 

b+ a-x 

d 

and has an extremum at * = l/2fa + b), which is a minimum or 
a maximum depending on the sign of <L At this extremum, the 
integral has the value: 



a/2 

d /giujj-j. 
-a/ 2 



Thus a scan made perpendicular to a line of width a and 
intensity c+d on a background of intensity c has a maximum 
(oi* minimum) whose value relative to the intensity somewhat 
away from the line is: 

d IgfuJ-du* 
-■2 

Suppose a scan is taken across a pair of narrow 

triangles as in the left of Figure 3 r 4, In the figure, 

kLntl /intensity) is plotted for successive scans. In the right 

hand side of the figure-, the values of the maxima of each 

scan minus the background value are plotted for each scan as 

v." 
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WIDTH OF LINE SCANNED 
1 UNIT = 1/2000 VIDI FIELD 



Figure 3. 4 
€0 



a function of the spatial Location of the scan along the set of 
triangles. Since the ■width of the triangle at a particular scan 
is a linear function of the position of the scan along the 
triangle j the plot just mentioned may be considered to be a 
function of ihc width of the line being scanned. Hence the 
x-axis in the plot is in units commensurate with this distance 
measure, and in the same scale as is used in Figures 3. £ 
and 3. 3 relative to the physical dimensions of the vidisscctor. 
The values plotted are thus those of kLnU/mlx}) - kLn(l/c) p 
where m(jcl is the actual minimum (since the target i& dark) 
of intensity along a. scan of a line of width x, and c_ is the 
background intensity. If the blurring function behaves 
as: curding to the theory proposed, we should have; 

x/2 k/2 

c - m(x} = d / g{u)du = 2d/g(u}du (5.2) 

-jc/2 

where d is the absolute value of the intensity difference between 

the background and the target. Using the curve of the integral 
of the function in Figure 3. 3,. and doing a certain amount of 
arithmetic involving logarithms, we can obtain a curve of 
predicted maxima in accordance with the premise expressed 
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In (3. 2), and with the assertion that the curvet in Figure 3, 3 
represent g[\} and its Integral. This curve is indicated In 
Figure 3. 4 as the dashed line superimposed on the plotted 
curve in the figure at tb.v right. The closeness of these two 
curves indicates good consistency between these two approaches 
to finding g(x); and provides further evidence that the curves in 
Figure 3, 3 represent gix.) and its integral. 

A third approach to the computation of g(x) is based on 
the fact that at a wide edge, the value of the intensity along a 
line normal to the edge is related in a simple way to the 
function; 

y 
My) = Au)dx. 

— dfi 

In particular, if the intensity function for a vertical edge is 
given by: 

IU) = a x < 

Mx) - b x > 0, 

then the distorted value of intensity, I*{x), is given by; 

00 00 



IHx) ■ / E(a£-u)g<u}du = a 4 b /g(u}du. 
-■ -jc 
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This equality is derived by an argument similar to that given 
above> In Figure 3, 5 we have the VEdueS of JsLnU / intensity ) 



o POINTS OBTAINED FROM THE 
INTEGRAL CL'HVK IN FIGURE 3.3 



L,SCAN 
LINE 



INTENSITY PROFILE AT AN EDGE SOURCE 

1 UNIT - 1/2000 Vim FIELD WIDTH OBJECT 

Figure 3,5 
for the intensities along a line normal to an edge, subject to 
the blurring of the vidissector. The illustrated points along 
the curve represent valuer obtained from the integral curve 
in Figure 3* 3, Suitably Scaled. Fur clarity, the curve 
represented by these points has been omitted. The close 
correspondence between the points and the curve further 
verifies that the curves of Figure 3.3 are a good approximation 
to g{x) and its integral* 

III. NOISE AND THE COMPUTATION OF PU^I ) 

A given noise 'free sample l^ turns into a distorted 
sample I* as shown in the previous section by the convolution 
■of I. with some blurring function. The conditional probability 

63 



P(J.j/L) may easily be expressed as the probability that simple 
noise on the sample I* would give rise to the configuration of 

intensities J,, 

■i 

It has been claimed that a large number of intensities 
read at a given point under a given lighting situation by the 
vidisKftctor have approximately a normal distribution. 
According to the way in which the vidlssector is construct ed, 
the absolute noise level is a constant for the logarithmically 
scaled intensities delivered. Actually there are four possible 
noise levels; however all work in the remainder of this paper 
will involve only one, the most accurate. Hence we shall 
assume that there is a. single constant noise level* The 
intensities read at a particular point for a particular set of 
lighting conditions follow the distribution; 

where a is a constant. From this may be computed the 
probability of a given configuration of intensities 

v ,* v ,*> ■■- i v at points with actual intensities 
l 2 m 

u. u ... r u . Since the noise -induced deviations are 
l m 
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Statistically independent, this probability Is 



i /j H a J 



n 



1=1 
or, letting 



"lj-^^V* (3 ' 31 



'.'.■■::|-, 



k 



we have: 

--Ft-"! 

The value of a for the signal to noise ratio has been reported 
(Horn 19&9) in terms of the percentage error of a logarithmic 
intensity value corresponding io an error of one stand ard 
deviation of the errors at that intensity. Again, this 
percentage is a constant for all intensities, and is reported to 
have the value 1. 5%. Since the logarithmic intensities are 
given by: 

ioo a Log 2 U/a 

the absolute error corresponding to One Standard deviation 
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is: 



a > 64. Log Jl- JD16I) - 64. Log 2 (I) = 1. & <3. 5) 



This id the value of & which will be used In (3. 4)< 

The procedure for computing the conditional probability 
P^JJSJ/I^S)^ is to determine the blurred version I* tS) of the 
set of Intensities I.{S) h and to calculate the value of 
P{J,(S)/I*(S)} by (3. 4). The relationship between 1.(5) and 
^(Sj is in general governed by the function f(u p v) referred to 
in the previous section. However we shall consider only 
cases where (3. 1) applies, and will need only the function 
glx) to determine this relationship* 



IV + A MODEL FOR THE SET {l*j AND COMPUTATIONAL 

CONSEQUENCES 1 



In this Section we shall give models for the sets 
jljMS)j c-f noise-free distorted intensity configurations taken 
over particular rectangular regions S, Out attention will he 
confined to sampling regions S which have a particular width, 
and whose lengths are integral multiples n of the width. We 
may consider such a. region to be composed of n Bquart 
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sub -components, denote it by S , and use ihc definition: 

I*{S > = I*!\ 
In l 

Hules will be given for assigning a priori probability values 
P{I*. ) to the elements of \l*'A - It will be recalled that there 
is a determ.ina.ttt 1:1 correspondence between elements I*- 
and elements L(S J, Thus the analysis in section I of this 
chapter remains valid with I* j in place of I.(S). It follows that 
th« value of Q{ J(S)} of E2 + 1) is determined by (3* 4} and by the 
values of P{I*j ). The latter are the subject of this section, 

Consider, for a fixed n, the set |l* I of all noise -free 
distorted samples over rectangular regions composed of n 
square sub -components. We shall divide this set into four 
subsets as follows : 

Let CLi be the subset of |l*-'| consisting o£ sets of 

intensities from Lines in the visual field exactly 

cents rei in the rectangular region. 

Let CE be the subset of [ p^ } consisting of sets of 

intensities from edges in the visual field exactly 

centered in the rcctiiJiguleX r-Ligicn. 

Let CH be the subset of Jl+. j consisting of sets of 
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intensities from a homogeneous area in the rectangular 
regipn. 

I*„ i consisting of sets of 
intensities from lines or edges which are not centered 
in the rectangular region. 

Wc shall first consider a model of CL, and base it on 
the following a^au m ptiona ; 

1) That the absolute intensities of the samples of CL 

are of no consequence, so that in each square 

s ub- component of a sample I* n the sum of the intensities 

may be assumed to be 0* 

2} That or. any section across some j-th sub -component 

of a member I*> of CL, the intensity function has the 

form; 

where &j j will be termeij the relative amplitude , and 
* J 

b. . ifl choaen to conform wfth assumption 1), 
3) That the relative amplitude a? ^ of an intensity peak 
(or valley) is uniform within the j-th sub-component of 
I*! 1 . A sample I*. may thus be described in terms of 
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an n^tu pie of relative amplitudes (a. ., r + 1 , a, ), e.g. 
as I*(a- fJ ... « lu tt ]i. 

4) That each sample I*. was given rise to from some 
line, highlight, etc., In the real world with an 
"idealised" relative peak {or valley) amplitude a-, 
which is uniform along the length of the line. 

5) That this idealized relative amplitude is purturbed 

randomly along the length of the line so that there is a 

minor fluctuation of relative amplitude a- ■ among the 

various sub -components u£ a sample 1*^ t with a 

2 

normal distribution with variance <r . 

n 

The fir'Sl Of these assumptions ignores a certain amount 
of information about a line which is useful in distinguishing it 
from a non-line, Hruadly speaking, it prevents a distinction 
between a aeries of peaks in aline whose absolute intensity is 
constant or regularly varying; and a series of peaks in a line 
whose absolute intensities vary randomly or dis continuously. 
This is not undesirable in the case of a line lying in the real 
world partly in shadow. However^ it would be desirable to 
eliminate as lines a aeries pt "lined up' 1 intensity peaks whose 
absolute intensity varied randomly, as this would likely he due 
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to noi^e. 

The second assumption is justified on the basis of the 
discussion in the first section, and by the fact that we are 
considering noise -free but distorted samples. 

The third assumption is justified c-n the basis that the 
"ridges" of intensity of samples in CL are rather uniform 
along their whole lengths, and would thus be quite uniform on 
square sub -components. This assumption is particularly valid 
if a square sub-component is only sampled along a line 
traversing it perpendicular to ihe orientation of the rectangle. 
This is approximately what is done in the program which will 
be discussed in the next chapter. 

The ''idealized" constant relative amplitude of 
assumption 4} maybe thought of as the relative amplitude 
which would be obtained in the absence of physical defects of 
the object being viewed or on the vidis sector photocathode, and 
in the absence of spurious local reflections- and anomalies of 
lighting. The purturbations of assumption 51 may be thought 
of as a result of these defects etc, , e. g. a nick c-n the edge of 
a cube, a slight shadow or a bad spot on the photocathode of 
the vidissector. The idea of an idealized constant value of 
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the relative amplitude along an edge Is further justified by the 
fact that intensities used will be logarithmic. Any ridge of ' 
intensity, for example a highlight, is a relative contrast 

phenomenon, and may vary in absolute uitcnsity, but not in 
relative intensity. A logarithmic intensity function would tend 
to register amplitude of highlight the same regardless of the 
absolute intensity. The same is true when the argument is 
applied to intensity discontinuities at an edge< 

It follows from these assumptions that if we let F{s.) 
be the a priori probability of an edge in the real world with 
"idealized 1 ' intensity a, then the probability of ex is tenet of 
a particular I*, = I#la- ,,...,!,_) given rise to by an edge 
with idealised relative amplitude a_ and parameterized by 
amplitudes a, , + . . , aj is given by: 

n _ i. r a--a ;j y 

FU 1 I"^t ^ l C " da ifl ... da^. 
j-1 

The probability that this I*j would, occur from a source with. 

any idealized amplitude is given by the Sum over & of the above 

expression, This Sum is simply the a priori probability P(I*. ) = 

This sum may then be evaluated provided the distribution 
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F(a) is known, Assuming it is normal with standard 



deviation p , we have: 



™-mi-e*^ 



da, 



when.- j^it'T.) iy tins a priori proijubiliLv ■:;■: v.>:i^tr-icp of 
a member of CL. We then have: 



da da. . ... da, J3.fi) 



P(CL) V^ ^ 2[aJ 2L "J J 

This sum, including the differ ential da may he expressed as 
as integral, with the exponent slightly rewritten: 



/ 



> _I|V + na* - 2a La^ 4 ^ af, -[ 



The exponent may be further rewritten as; 



Letting: 



A - sr*-pr > 



and completeing- the square, the exponent becomes, in 



terms of A and B: 



-| [AU a -3a(B/A)+ lB/'A>*> - B*M + £a'Jo*l 



The integral becomes; 



' = -A/2{a - B/A) s - 1 /2(Ia 4 /*£- E 2 /A> 

* dft. 



/« 



The second terra in the exponent is independent of the 
integration variable a^ and may be taken outside the 
integral sign. The remalninig definite integral is well 
known to have the value^pr/A. Thus the integral is 
equivalent to; 



ie 



I Til 5.1 

a |_ ^ " aJ 



The result of substituting in this expression the values 

of A and B, of substituting the resulting expression into 

(3. 6}, and of doing some straightforward algebraic m ampliation 

is: 

Pft * ( \,l' *" \* )B 

P(CL) g> p'lbfrnti* vHoi+tyft I ... ,. 

V2Wtf? V^^ C da i, i- -^i M V- 7) 



An analysis of the CE category may be made in a nearly 
identical fashion. In this case* the function g(x} of assumption 2 J 
is replaced with the function h(x), its integra.1. Ak was paint ed 
out in Section II, this function gives the intensity along a normal 
to an edge, and Lb the obvious correspond to gtx), which gives 
the intensity along a normal to a line. A formula for the 
a priori probability of a member of class CL may be derived 
from an analysis nearly identical to that given above r The 
resulting formula differs from {3.7) only in that different values 
appear for the variances cr n and p nt and that P(CE> is used in 
place of PICL). 

Again, an analysis of the CEI category may be made along 
nearly identical lines. In tJniid Ease, the function g[x) ;y replaced 
with l(x) = ax -r hj the function which gives the Intensity profile 
across a homogeneous region. Again the resulting formula for 
a priori probabilities of members of Cti differs from (S + 7) only 
in the values of p rt and p„j and in the use of P(CH) in place of 
P(CL). 

At this point it is convenient to recapitulate tome gf the 
foregoing by rewriting the formula for Q(J(.S)) appearing at the 
end of Section I in terms of the specific notation introduced in 
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later sections, and Some additional new notation First, ft will 
be recalled that the nutation I*(a, „ .,» M a- )> was introduced 

1| 1 lj Ij. 

to refer to noise-free distorted samples over a region con^i^ting 
of q square sub -components,, and containing a line centered in 
the region. A value a, . refers to the relative amplitude at the 
intensity profile of the i-th sample, taken across the j-th 
sub- component* We may similarly adopt the notation 
L*{bj .., ... , b, ) to refer to noise-free distorted samples 
from the class CE, and I*tc^ . , . r r , c . J to refer to samples 
from CH. We shall reserve the notation I*, to refer only to 
distorted noise -free samples from the class CS, We may 
introduce similar notation for noisy Samples by a definition 
similar to that used at the beginning of Section IV: 

By this notation is meant the i-th possible noisy sample on a 
region consisting of n Square srab-componeots. 

The sets CL., CE^ -CH and CS provide us with an 
alternative for expressing the function C^LtS)) used in the 
formula for Q(.J(SH in Section I. If we make the following 
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definitions : 



■I 



5><jJVl*(a^ j a i,n 3)P(I * <a i, I' "' ' a i,n J) ' {3A> 



CEUJ 1 ) 



yptjf/r*!b. b- n i)PU*tb. ., -,♦ , b> )\ {3, 



E?> 



CHUJ 1 ) 



CS(j n ) = YP(J n /E* n )Pa^)- (8. U) 

Then it folLows from the 1:1 correspondence of distorted 
noise -free elements, and ndn -distorted noise -free elements 
that: 

^pgj'/i^^pa.^)] = cuJ* 1 ) + ceu^), 

G< 1^5^1=1 

and 

Y P(j"/I.tS ))P(U3 J) = CH{J?) + CS(j"l. 
^ J 1 n l n J .1 

G(I i (S n ))=0 

Consequently we may rewrite the formula at the end of Section I 

CL(j") +■ CE( J, ) 

= *T 7^- 1 ", «.A (3.12) 

CLU )* CE(J ;j ) + CH(J.)+- CStJ ) 
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We may now turn to an analysis of the class CS, It turns 

out not to be necessary to mako an explicit model aa was done 

for the classes CE, CL and, CH. To see this + recall 

Formula (3. 12) of the previous paragraph, and Formulas 

VS. B) - {%* 11), which define its components. An element in 

these latter sums, e. g. t 

P(J™/I*(a. ,, ... , a. ))F(I*(a. a. )> 

] _. i ij n i, i ij n 

from the eumCL, may be thought of as a, measure of 

JL 

similarity between J. and the distorted noise -free sample 
^ a i 1 J " " " ' a i J' weighted by the plausibility of the latter. 
Thus the value of QS, for example, Is roughly proportional to 
the plausibility or frequency of occurrence of members of CS, 
and to the similarity of J, to members of CS. 

If J? occurs over a rectangle £ on the visual field 
which is centered on the Linage of a line^ the value of 
CSU? ) is quite small relative to the other members of the 
denominator of QQ- ), owing to the relative inlrequency of 
members of CS as compared with members of CH, and to the 
fact that such a J, is as similar to a pattern of intensities 
from a homogeneous region as it is to a pattern of intensities 
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containing a. skewed line. Since the value of CS(J. } Lb thus 
small, It could be omitted in this case, or at lea^t approximated 
with the sum of values of 
CL k (j") + CE k tJ n ) 

J J 

k 
over a set of regions S which are nearly coincident with S n , 

but are slightly skewed with respect to it. These values may he 

obtained "for free" for any region S by a line-finder which has 

to be able to find lines in all positions in the visual field; since 

it would necessarily compute CL {J. ) and CE (J. ) in the 

j j 

process of testing the other regions $ n for the presence of lines. 

If S is a region in the visual field which did not contain 

n 
a line, a noisy sample J. over 5 would, by a similar argument, 

have the property that CStJ ) makes a rathe i- s in all contribution 

to the denominator of Q( J. ), and could be similarly approximated, 

or indeed omitted. 

It is only for the case that the region S n contains the 

image of a fcKtw line that the value of CSt J. ) becomes 

significant. However, this effect occurs only for rectangles 

approximately coincident with a rectangle S n centered on a line 

in the visual Held. Let Q*(J?> be the same as Q(j"") but with 

J J 



the value of CS(J. ) omitted from the denominator: 

J 

CL(j") +■ CE< J™) 
Q*(j") = J I (3* 13) 

J cuj"} +■ cee/ 1 ) 4 chu") 

J J J 

It has been empirically checked that Q*tJ. J obtains a maximum 

value locally approximately on the tec tangle S n centered en the 

n 
line. This is quite plausible, since ibe value of Q*(J. } should 

be large for rectangles approximately coincident with S . and 

if it didn't have a maximum on S , it would either he offset 

parallel to 5 n . which is unlikely because of the symmetry of 

the intensity pattern, or there would b« several mmma, 

n 
which seems not to happen , Hence the omission of CS(J. ) 

j 

from the denominator of Q(J^ ) has the effect of giving 
erroneously high values on regions nearly centered on a true 
line, but not erroneously high values on the maximum value 
region, namely a region centered on the line t However, these 
values are not so augmented as to create false maxima for 
regions not quite centered on the line. 

In summary, the set of noise -free distorted samples 
is partitioned into four classes CL, CH, CE and CS. Our 
threshold function QU n ) is expressed, in {3. 12 h in terms of 
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fern * sums CL(j\ CH^J 11 ), CE(JJ and C5*/ 1 ) over these four 

J J J J 

classes. Explicit models qf CL, CH and CE allow a simple 

explicit expression of the a priori probabilities of members 

of these sets. These probabilities, together with the results 

of Sections II and III; determine the values of the sums 

CL(J?). CH(J. ) and CE{J ). It has been fib awn that CS(j n ) 
1 J J J 

may be computed indirectly, or even omitted from the 
denominator of Q(*T ). The latter is permissible, resulting in 
Formula (3. 13), in the case where the regional predicate i£ 

applied systematic ally over the whole field of view. 



V. COMPUTATIONS OF CUj"), CE(J n ) AND CHtJ 11 ) BY 

MEANS OF INTEGRALS J J 



In the previous section* it was pointed out that the 
threshLild function QtjJ 1 ) eemld be approximated with the 
function Q*( J ) given by (3 r 13). This function is determined 
entirely by the values of sums CL(j"), CE(j"} and CH(j"), as 
defined by (3, 8), (3. 9) and <3. 10*. In this section, we shall 
show how to express these sums as integrals, and tu reduce 
them to relatively simple closed form. 

It might scum a gnnd strategy to evaluate the 
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expression,, for example CL(.J. ) as given by t3 r 6), by 
subsLiLuting i values given by {$+ 4) end (3< ?) respectively for 
the two terms within the summation sign, expressing the 
resulting sum as an integral , and finding a closed form for 
the value of the integral. This approach appears to run into 
difficulties, and a slightly different, approach will be used 
instead. 

Our first step is to express , e, g r t CL(J. ) in a more 
udnverdent form. Since the sum in (3, &J is taken over all 
n -tuples, CL(J^) may he expressed as; 

a. . a 

a. 
i, n 

where the sum over a. , means that a, ^ in the summand 
takes on all positive integral values < We may express 

PW^! ■!■*»" 

2]P(I*(ai ,i a ift )/ajP(a), 

a 

where P(a) is the- a priori probability of a line with 
idealized amplitude a.;, and PftHa. ,, ... , a. )/a)isthe 
relative probability of occurrence of a noise -free distorted 
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sample with sub region amplitudes a. ,, tll t , given 

a Line with idealised amplitude a r The conditional probability 

FU*(a^ 1 , . rT , a^ n )/a) may be exp^cfised, as In Section IV, 
ajs a. product of prob shinties: 



ii 



where P(a tjk /a} is understood as the probability, given a line 
with idealized amplitude a, that the k-th sub-comppnent of 
a noise -free sample will have a relative amplitude a. , r The 
above probability may be expressed aa a product, since the 
elements in the product are statistically independent. 
Likewise, by statistical independence, PU? }!*{&; , , . . . , a- )) 
maybe expressed as a. product; 

™*i m \i 8 i. n ) >=fl I '( J >>' a i,k'. 

k-1 
where J^tk) is understood to mean the set of intensities of the 
noisy sample j" which He within the k-th square sub -component 
of the region. Substituting and rearranging, we have: 

CUJ P = 

Z^I-Z I> J >^i,k^i 3 kM- 

a a a, k- 1 '' 
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It is not very difficult to see that the product over k and the 

Summations over the a. i/s may be interchanged, so thatr 

CLU"I = 
J 

n 

Similar expressions exist for CE(J, } and CH(J. ) r 

■ J 

An expression for P(J. (k)/a. ) may be obtained by 
adapting (3. 4) to the present context, using the value of a 
from (3- S)^ 



L Di| 



*. 



n i 2 (1. 5/i m r % 



where 



i, ],k fr\ l > k D.k 

which is similar to (3- 3) in that it refers to some i-th noise- 
free distorted sample and a ]-th noisy sample, but has a 

third subscript to indicate it involves only the k-th 

sub -component. The values Uj ^ ere the m intensity value-s 

within the k-th syb -component of the i-th noise -free distorted 

sample containing a line; and are taken over points which may 

t 
be denoted by p^ ^ Since the amplitude in this region ie 
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a ijk i *e have by assumption 2) of Section IV that the intensities 

t 

at points p^ k are given hy: 

where x^ k is the perpendicular distance between p\ k> and a 
line traversing the center of the region in the direction of the 

expected line. If the geometry of the points p\ fc is fixed for 
all sample and sub -regions, then the value of g(x* ^J is 
independent of the sample and sub region and we may 
exprssa such an intensity value by gCjc t K By assumption 1 ) 
of Section IV, we may choose b i k so that the sum of u| j, is 
zero, L. e. , by choosing b._ k - g, Hence defining values 
(U j U^Jby: 

t 

we may express the m intensities in the k~th sub -component 
of the i-th distorted noise -free sample by: 

* y i,k j ■ ■ ■ ■ S^k* = 

ta^U 1 , ... , a^fcU 1 ") (S.ig) 

wherei 

£- u = °' (3, IB) 



l 



OS 



To be consistent with assumption l), we snould also assume 
that: 

£v* jk =0. (J. 20) 

The value oi D t of (3. 151 niay be expressed in terms of the 
ij Ji it 

values just defused as: 

It is easy to show that: 

^j.k-W«4,k-*j,k> Z * B j,k' a22) 

where: 

Z IlV 

a * r ^ J* k (3,33) 

b - EtuS 3 13. 24) 

l 

B.,=ri>f. ,,"*! j^V- (3.25) 

i 
Combining we have, omitting the differentials dv^ k : 

l r h<a ;k - a» t ) a ■» R|j 1 

We may expretis P'^ ^./a) as before by assuming a 
normal distribution on the various values of aj k . with 
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26) 



standard deviation q and mean a: 

n 



«*U'»»- 



V^ n V n 



e 



& - a. 



Jd \ 



k- 



(a.27) 



Combining (3, 14), (3. 26) and (3* 27) we have, again omitting 



differentials dv 



j*t- 



n 



^ 



2[ (1.5)' ^ +_ ^~ 



da^da (3. 33) 



with b p a=^ ^ pnd R^ k given by (3. 23), (3, 24) and (3. 25). The 
inner sum may be put in the form: 
-Ba, t Ca. . 4 D 

which admits o£ the approximation used In the preceding 

section; 

-Bx * Cx + D 

Ale ^^ 



/,_ -Bx + Cx + D — tD 



— ce 



VB 



Applying this approximation to the inntr sum in (3. 2S), we 
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have after considerable algebra: 

n 



1, 5 



be? 



iTr^ ^ b<afr - a)' 1 
g 2 [(1,5) ! (1.5)? - hv* n J da 



where again, the diffentials 6v have been omitted, 
may alternatively be expressed: 



This 






1.5 



HSw i. srya.sj? +-b^. 



£ 2 U-L, S) a (1. 
and expressing P(a} as before by; 



i) ! tbfl 



2 

n — ' 






V^ff Pn 



p„ 



n. 



this latter express ion for CL'tJ- ) may be put in the 



form: 



-Ba^ + Ca + D 
a 



•d; 



and may be approximated as before to yield; 



CL(J n ) - PtCuf- 
L 






\W2^U5T^l. 5} 3 1 tiff,! 



^ 2 H 1 . 5 >* (1.5 F+bctf+nbrf ( b(*? +u . & V H b^+tibrf-K 1 , 5 ) a J | 



,(3. 29) 



with differentials dv. omitted. Similar expressions e:x.Lst 
for CE(j") and CH(j">. 
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vT. APPLICATION OF THE TlIEOHY TO SOME ACTUAL 

VIDISSECTOR OUTPUT 



In this Section we shall apply some computational 
procedures suggested by the foregoing theory to intensity 
values obtained from the vidtssector. These intensity values 
will be taken along single lines normal to various lines and 
edges in the visual field, The results of the compulation 
will serve to validate the theory presented, and justify 
some assumptions and approximations in its computational 
implementation which will be used by the program described 
in the next chapter. 

In applying the foregoing theory of a regional line 
predicate, it in first necessary to choose a suitable physical 
width for a rectangular region. It appears that, for the 



vidis sector in current use, this width should be between 25 
and 50 units, where one unit is 1/2000 of the full field width, . 
This range of values is chosen to conform with the assertion 
that, in the terminology of chapter two, the majority of 
regional evidence for the existence of a line Is contained 
within a neighborhood of the line whose radius is on the 
order of a. few times the "critical resolution radius" of the 
vidlssector. This latter quantity, as defined in Section I 
of chapter two, may be easily obtained by examining 
Figure 3. 3. It may be seen that the intensity in the upper 
curve falls to one half of its maximum value at a radius of 
three units from the center of the curve. This value is 
approximately the critical resolution radius as previously 
defined. For present purposes., we took "a few times 1 ' to 
mean specifically eight times, which yielded a region width 
of 50 units. Later results, reported in the next chapter, 
indicated that this value may have been conservatively large > 
and somewhat better results seem to have been obtained with 
a region width of 2$ units. In any case, the value of SO units 
turned out to be adequate for present purpurea. 
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It seems appropriate to begin an empirical investigation 
of the properties of the regional line predicate here developed 
with a consideration- of the case where the region -o which it 
is applied consists of a single square sub -region. This 
admits of a simplification in the formula for Q* C.J. ) as given 
by f3 r 13) and {3. 29). In partir.uic.rj the term involving 
(a* . - a* J in (3, 29) becomes zero, since a* is the 
average of only one term, Thus this term is eliminated from 
the exponent in {3. 2&)i and likewise similar terms are 
eliminated from corresponding expressions for CE(J^) and 

Another simplification of the exponent in the expression 

n 
for CL(J. ) given by (3. 29), and likewise for corresponding 

J 

expressions for CE(J. } and CH(J }, follows from a 

* ) 

consideration of the relative magnitudes of the values 

2 2 

{1,5) and ber The latter is the value of b given by (3. 24) 

times the variance of values of idealized relative amplitude 

of the various lines in scenes in the real world. Since the 

distribution of these values has a mean of zero^ it follows 

from elementary properties of the normal curve that about 

1/3 of all lines have idealized relative amplitude a with the 
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property that laJ > a- . Since the variance of the sub- region 

amplitudes a. - Is small relative to the variance of the valuta 

Lj 'J 

of £, the same holds true -of these amplitudes, i. e, , about 1/3 

Of all Sub -components of noise -free samples containing' Lines 

have amplitudes a, , with ia, .1 > <r , If the intensities 

*■» J 1 jJ n 

within a sub -region of a noise -free distorted sample are given 
ty * u i, k> ' ' ■ a v i k* and theip spPgad, is defined aar 



^"U • \S- 



with 

i 



then it Is easy to see from (3. lfl) that this spread has the 
value a- */¥, where b_ is as defined in (3. 24 K It follows 
from what wee said previously that about 1/3 of all sub- 
components of noise -free samples have a spread of greater 
LhanV&V, More informally stated; y/bv is a not uncommonly 
large value of the spread of intensities in sub -components of 
noise-free samples containing lines. In the case of noisy 
samples containing lines,, empirical investigations suggest 
that a spread of 3 is not uncommon; and it is easy to see 
from the fact that noise level is relatively much smaller than 
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2 2 

bff » (1- 5) . 



30, that this value Is also not an uncommonly large value of 
spread fnr noise- free samples containing' lines. Consequently; 

(3. 30) 

Wow by simple algebra the exponent in (3, 29) may be written, 
excluding the third 1frm which is aero, and using the relations 
in (3 H 23), (3.24) and (3.25): 



(1,5)* " (1.5ft(i.s7^-bff*+nbp!)_ 

It follows from (3. 3 0) that the coefficient of the second sum 
may he approximated by (l/b<l. 5) a ), so that the exponent of 

{3. 29) is almost exactly equal to: 



?"U 'frfrj.i' 1 



(3.31) 



(1.5) b(1.5) 

A similar sort of simplification of the coefficient of £ 

in (3, 29) may be made from a consideration of the reLative 

2 2 2 

magnitudes of nbp and bff . The value of* represents the 
c n 

variance of a quantity a, idealised relative Intensity; and 

p Is, according to assumption 5) of Section IV, the variance 
n 

of small perturbations of the latter. It is reasonable to assume 

2 2 2 2 

that c » P , and that even bff >> nbj> . By this approximation 
n n n n 
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it follows that the cu efficient in (3, 29}- is close to: 

Ft CL) . (3,32} 

■Vb o„( V2trl . 5 )" 
If this and the approximation in {3. 21) are applied to (3, 29), arid 

similar approjiim aliens are applied to corresponding formulae 

l i 

for CEU ) and CH{J-); Formula (3. 13) reduces to: 

1 SLfcfl I SE(i) (3,33) 

Ft CD v? 2(L. 5) a PjCE) ^ 2(1. 5) J 

*b >, g ^VE'g; g 

1 SUj) '"""' 1 gUj) 1 SHjjj 

PjCL) ^ 2 tl. 5?, P(CE) _ 2 (1. 5>« P( CHE „3 (1. &> J 

Where: ffj, ffj, and ffj are respectively the idealised relative 

amplitude variances of elements Gf CL, CK and CM; 

b^lrf, (3.34> 

t j m 

and tU , r . , , U ) is some paradigm intensity 

pattern for component regions of members of CL; 

b'--Zw t )i {3.35) 

1 

and (U 1 J . r . t U ,m ) is some paradigm intensity 

profile for component regions of members of CE^ 

h"=ZW*)\ (3.361 

t 

and (U" , r r . M U" m J is some paradigm intensity 
profile for component regions of members of CH; 

J) a 



s:.{j) ■:.Eu t vJ jl ) 2 /b (3. a?) 

SE(j)= (^U'V^^/b 1 (3.3B) 

SH(J)- {f TJ'^j^/b" (3.39) 



and {v. , . ... , v. ) art.- thu intensities in J. 
]► 1 J* I J 

It is necessary to assign values tu the remaining 



constants in {3. S3). First we shall assume that lines and 
edge£ are equally likely, SO that P(CL) - F{.CE). AifiO in 
the examples which follow, it will become clear that an edge 
or line occurs in tint* sample out of 75, so that; 

!/75P(CH) s P(CE) = P(CH). 
Secondly, it follows from empirical investigation of the 
range of spread of homogeneous samples with various 
gradients, that five is a reasonable value for b'VJ r It was 
already pointed out that 30 is a reasonable value for h&. r 
and we: shall assume this is the case for b'ff 1 as well. Thus: 

Q*(jS 

J 

SUli SE(j) 

2(1.5)' 2(1. S) 1 

e e 

(3.40) 



SL(j) 5E(i) SHli) 

2(1. 5>* 2(1. 5) a 2(1. 5) J 

£ + £ + 450 £ 
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For values greater than , 5 this function is near Ly- 
nn nnotOne With: . 

Max{SM])/2£l,5) 2 , SE(j)/2(l. 5) 2 ) - 

SH(])/ii(l-5} 2 - 1,11(450.), 
or equiv&l*ntly> with the function Q**(Jj> given by: 

Max(SL{jL SE{ j}> - SH(j> - 2 7. 5 (3. 41 » 

This follow a from an observation rir.rAM ihc: res*-.] Its givi^n latur 
in this section, namely that SL[j) and SE(j) are rarely of the 
same order of magnitude. This is not surprising, in view 
of the fact that one expresses the similarity of the obtained 
pattern to a noise fre£ adgc, and the other expresses the 
similarity to a noise- -free line. For values of Q*(J7) 
greater than .5, the pattern m«fit he at least somewhat 
similar to either an edge or a line. It could not be 
simultaneously similar to both* Consequently the formula 
(3. 40) above, which is Of the form 

A -i- H 

A + B + C 

has the property that when its value is above . S f A and B 
are; considerably different in value, and both positive. 

95 



It is not difficult to show that in such a case if, for example 

MazlA, B) > 10Min(A, B), 

then the approximation: 

MaxiA* B) A+ B 

MwrtA, B) + C AfB + C 

is good to better than 6 percent. Th* fact that QH& is 

J 

monotone with Q##U .} follows from the fact that 

MaK(e a , e ) 
Mas(e , e } +■ e 

is monotone with SMax(a, b) - c, if a, b and c are positive. 

It is desirable to apply the predicate to a variety of 

regions which are homogeneous, contain off -cent ex lines, or 

contain centered lines or edges. We have chosen data from a 

series of areas on the visual field which are 200 units wide 

and contain ibe in-.ages of vertical edges and lines approximately 

centered laterally. The predicate will he applied to 75 

successive overlapping regions 5C units wide, which are 

offset relative to each other by two units and thus exactly 

cover the 2 00 -unit -wide area. The procedure will he applied 

tu vertically oriented regions only, since only vertically 

oriented lines exist in the data samples. The fact that 75 
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regions are tested and only one contains a centered line or 

edge justifies the relation F(CH) = 75P{Cii:) used above. 

Finally j before applying- the predicate to real data, 
which will be done by computing the function Q#*(J ) 4 it is 
necessary to give values for the sets JU [, {u 1 \ and Ju" J 
used in formulas {3. 34) - (3, 39). It will be recalled that 
these are paradigm sets of intensity v sluts, at points in some 
fixed geometry within a square sub-region. They correspond 
respectively to a line centered in the region, an edge centered 
In the region, and a homogeneous region. According to the 
fixed geometry wliich we shall use, the points will be equally 
spaced along a line traversing the square a ub -component 
through its center and oriented normal to the expected line. 
These values may be ordered in an obvious manner according 
to their positions along the line mentioned. They are shewn 
graphed as a function of this ordering" in Figure 3. 6. The 
values Of b, b 1 and b" for these sets of intensities are all unity. 
The intensities from the real world will come from single scans 
of 100 points taken normal to a vertical edge or line in the 
real world. The points will be separated laterally by two 
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units, so that the total width of the area scanned will be 200 
unite. If it is desired to apply Q**(j|) to some region along 
this scan, out of the 100 intensities i L , r . , t i; some 

set I i+1^ ■ ■ ' ' *i+2B fal1 with i^ * he region.. Then it la 

trivial to select intensities from among these to be used as 

1 m 

the values v^ J# ... , v ^ for use in formulas (3. 37), 

< 3. 30) and (3, 39). They are simply 1,^, ... r respectively . 

Thus we will be applying (3.41) to 75 successive 25-tuples from 
the 190 Intensities in a particular scan. 

It is important to digress here to point out the 
advantages of the soft of geometry of points within a sub -region 
which is used above. This advantage will he realised in the 
next chapter when it is desired to apply the: regional predicate 
exhaustively over the whole field. In particular it will be 
desired to compute the predicate for lines of all orientations 
through any point P on the visual Held. Consider those 
situations where the regions are oriented within + 45 degrees 
of the vertical axis of the vidis sector, and all pass through 
some point P; and consider only the sou are sub -region 
containing the point F, which will be termed the p-th 
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sub- region. We shall in chapter IV take intensities relevant 
to regions so oriented, along lines separated by approximately 

the width s of a region, and oriented parallel to the x-axis of 
the vidissector. We may thua assume that there will be a 
unique scan passing within s/2 of the point P; and; that the 
intensities J^ . . r , i ffl along its length within a distance s/2 
of F are exactly the ones from which the v^ will be selected. 
The procedure for selecting the values v\ from among this 
l.l-I is parameterized by the angle tf Whu:b -.!:<: region [uaL-tcid 
with the vertical. If the spacing of points in the fixed 
geometry of points is d, then one wishes to choose points from 
among the 1^ ... ( ^ such that their actual spacing is 
approximately d/Coa(0). If tlie spacing of the points along the 
scan line is Itself d, which will usually be the case In practice, 
then this matching of obtained intensities with those in the 
prototype amounts to "stretching" the former by some factor 
between 1 and ^2, Thus the various values of, e r g, , 

SEQj^ p ) ■ L, v . U , which are used in the computation of 

n 
Q*(jj ) for various regions containing P, differ only in the 

amount of "stretching" of one spatial axis relative to the 

other in the matching of intensity values of the scan against 
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those of the set j [J J. Denoting these various valuta nf 
CE(J. ) byCE(J, . 0), it turns out empirically that the range 
of these values is rather small; and in particular for |0 f < 45°^ 

the values are within s a mething like 2 percent of the closer of 

'\ i' 

CE(J n „, 10°}, CHJ > 30 a ). One may thus compute these two 

values only, and for any region containing the intensities from 

which they were computed, use one or the other in the 

calculation of Q*(J^) for that region. In fact we shall in 

Chapter IV actually compute these two values by stretching the 

spatial dimension oi the paradigm, rather than vice versa, 

This amounts to using two peaks of different widths as paradigm 

profiles for lines A and two edges of different severity of slope 

for paradigms for edges. It should be clear that if the fixed 

geometry of the points were two -dimensional, e. ,u'. oil ;=. 

square grid, no such simple accommodation for the various 

angles might be possible; and doubtless the amount of 

computation involving the intensities in each neighborhood 

would be far more than the calculation of two values. 

To retain to the application of Q** to real data > our 

i'jri=t example, in che upper left Of Figure 3. 7, comes from 

a horizontal 3 can across the forward edge of a cube. The 
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scan covers approximately one eighth of the apparent width of 
the cube. The cube is a solid metal block which lias been 
painted with white paint bo as to have a very uniform surface 
and sharp edge. In each of the eight figures, Lhe logarithms of the 
reciprocals of intensity are given as thE bottom of the two curve a. 
The absolute range of values is 11 units, where 04 units corresponds 
to a factor of two in intensity r It is easy to infer from the- physical 
properties mentioned that the slope of the intensity curve on either 
side of the edge is due to a systematic non-uniformity in illumination* 
rather than to a nOn- uniformity af reflectance properties. At the 
edge itself there appears to be a sort of "negative highlight 1 '(a 
local peak). This appears alyo at :h« re nr.fi i"H of "i-e RCariS u: the 
upper right and upper-middle left examples of Figure 3.7, which 
were taken from, the same cube; end is apparently a property of 
this particular edge. The 100 intensities in the example give rise 
to 75 values of Q** which are graphed above the intensity curve, A 
value of Q** is derived from £5 successive intensities consisting of 
the one just below it and twelve on either side. Thus a peak in the 
value of Q** should exactly coincide with a peak or discontinuity 
in the intensity profile. The horizontal lines in each of the 
eight figures are placed at distances along the Q-* axis whioh 
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correspond to values of Q^ which are indicated on the lines, 
A reasonable threshold value for Q* appears from an 
analysis of this example, and those appearing later in this 
section, to be about 0.1, A cutoff level of approximately this 
magnitude will be used by the program described ii] the next 
chapter. This cutoff value is chosen on the basis of its giving 
about ^o% false positives. There is no sjmpLe exact relationship 
between the Q* cutoff and the fal&e positive error rate; however 
tt follows from the first theorem of cnaptL-r two that if is the 
cutoff value for Q-, then 1-0 is an upper hound of the false 
positive error rate. This may be seen by noting that for the 
threshold procedure, the sum in (2, 3) gives the false positive 
error rate. Every element in the sum is some probability 
P(J (S>J times a value leas than l-£. Since the values of 
P(J(S)) sum to one, the sum in (2- 3h i. e. , the false positive 
error rate, is less than 1-J& Thus Lhresholding Q# at 0,1 should 
give a false positive error rate less than 0.9. However* the 
extent to which the false positive error rate ia actually less than 
0. 9 is an empirical matter. Evidently the various probabilities 
and ao forth are such as to give an actual value of 0. 7 in th.Ls 
case, 
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An obvious question now arises: hgw subtle may an 
edge or line be relative to the noise level in order to be just 

detectable, e r g, , with the threshold of 0. I, An example of 
audi an edge i£ given in Figure 3^ 7 -at the upper right. It i£ 
possible to make a general quantitative description of such a 
Line or edge by means of a value derived from the cutoff value of 
Q*#, Support: the out off value is 0,1; then we have; 



or: 



nr 



or; 



or; 
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SL(j) SE(j) 1 
p m.5V -2<1.5? 




Max 


r smj) se(j)' 
act. of 2(1.5? 


„ 2(1. 5? 
+450 V 



* 0. 1 , 



SLjij SE(j} ' 

2(1.5}' 2fl75? 

9. MhjcLC , t 



450. £ 



2(^.5)* 



Ma*lSL(jl/2(l. 51| SE(j)/2<l« Sfl + t-n(9) * 
SH(j)/2(l*a)'+ LN<450. )> 

MaxfSUj), SE(jJ) - SH(j> - 27. 5 * 2(1. 5) : Ln(9) = &.9 



Max(SL(j), SE(])) -.SE(j)- 1^6 



(3.43) 
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Since SL(j), SE(j) and SH(j) are positive, the mini mum value 
of Max($L()J, SE^jH subject to (3, 42) occurs when SH(j) = 0. 
This is the case when the net inten&ity gradient for the sample 
i£ zero; a fact which can be deduced from (3- 39) and a 
knowledge of same of the arithmetic involved with simple 
least -squares analysis. In this case for a marginally 
detectable line or edge, SL(j) or SEjj) has the value 17, 6. 
If the intensity profile follows the form, e.g. t of the upp^r 
curve in Figure 3. G; then it is not difficult to aee, by the use 
of {3, 37), what its actual peaK to valley range would be; 
Suppose the 25 intensities in the noisy intensity profile are 
given exactly byau, ... , al' . Then SL(j) for the Be 
values is, according to (3. 37); 
[a£oM| 2 

However, the denominator term, which is simply b as defined 
in (3. 34), was chosen to be unity for the set of Us in 
Figure 3. fi; sb the value of EL(j) turns out to be simply a 2 . 
This, it was agreed, is 17. 6j so the value of a is 4. 2. 
Thus a marginally detectable intensity profile of this shape 
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la similar to that fit the top of Figure 3,G, but multiplied by a 
factor or 4. 2; and thus has a peak to valley range of about 
three units. In the case that EE(j) is a maximum and the 
phenomenon is edge -Like, one may assume the approximate 
shape of the intensity profile is that given by the middle curve 
of Figure 3, 6. By a similar argument, ihe marginally 
detectable sample of this sort has a range, in this case net 
intensity diilerence, of about 1. 7 units. The central fea.tu.re 
of the ec an In the upper right of Figure 3. 7 has a value of 
Q* at threshold; and the amplitude of the peak at the center 
may be compared with the former of the two theoretical, 
values just referred to. The intensity range for the whole 
scan is 17 units p Irom which it is easy to see that the peak to 
valley height of the small peak in the center is about four units. 
This compares well with the theoretical value of three units:. 
Clearly the amplitudes of marginally detectable lines 
and edges must be considered in relation to the noise level, 
Noise may in general be due both to time noise, random 
fluctuations in intensity measured at a point under constant 
illumination; and to space noise, which consists in random 
fluctuations in intensity as one scans across a surface. A 
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combination of the two may be obtained from an analysis of 
successive differences in intensity of the above samples in 
areae a way from the known feature*. This analysis yields a 
standard error of successive difference* at about two in ell 
cases. Since the variance of successive differences is twice 
the variance trf individual values abmil a mean, then the 
standard error of the latter should be about 2/J5, or about 
1. fi. This is approximately the value of the standard error 
involving time noise only, as reported in (3. 5), Consequently. 
the space noise is in this case negligible, which 1b not 
surprising in view of the deliberate choice of smooth- surfaced 
objects. In any ease, the marginally delectable amplitudes 
of 3 units and 1. 7 units for lines and edges respectively 
should he thought of in relation lo an underlying noise level 
whose standard error Is 1- 5. For example, using the £5 
point geometry previously described, one may state that a 
marginally detectable peak has a peak to valley range of 
twice the noise level, and a marginally detectable edge has a 
net intensity difference of about l . 2 limes the noise level- 
Six additional examples are given in Figure 3, 7. 
The left-hand upper -middle scan was taken from the same 
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cube arid under the game Lighting conditions as the top two in 
the illustration. The other five were obtained from a Scan 
across the vertical forward edge of a second cube under 
various lighting conditions. The Lower -middle pair, marked 
NEDGE2A and NEDGE3A, were taktm under identical Lighting 
conditions, hut at different heights along the vertical edge. 
This cube is uniformly painted, but the edges are slightly 
rounded, accounting for the pronounced highlight tdip in 
[i-itensvty, since the values are actually Log( I /intensity)) 
apparent in some Of the profiles, particularly the ant marked 
NEDGE2A. 

There is a certain weakness in the use of s-o great a 
region width &s 50 units, namely that the ifcitetjsi:.\ profiles 
across a region may have non-negligible gradients t-nward 
the edges at the region, appearing, e. g, , like the profiles 
in Figure 3->&. For this reason, it is useful to generalise 
the thresholding procedure so far described by computing 
values SLjijh . - - , SL^j), SE^j), ... t SE m (j) for a set 
of n peaked profiles and m edge profiles, and using a 
maximum of all these values in place of the maximum over 
two elements as in 3,41, Denoting this generalized version 
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of Q** by Q'**- t we may express it in two slightly different 
forms : 

Max £.SL t (j) f SE k <j)} - SH(j> - 2 7. 5 = 

Ma« (SL (]) - SE(]), SE k <j> - 5E<j)> - 2 7, 5, (3, 43) 

l.k 

This generalised formula will be used in the program described 
in the next chapter. In the remainder of this chapter we will 
discuss its application to the previously analyzed data of 
Figure 3, 7, 

For present purpo&ee, we shall eon&ider only a possible 
multiplicity of paradigm peak profiles. Four additional 
paradigm profiles besides the one in Figure 3. 6 are illustrated 
Lri Figure 3. a, and dc:r.o:c:d hy WPKAK, CPEAK, BPEAK, and 
APEAK respectively. It is instructive to compute individually 
the values whose maximum gives the value ofQ 1 ** according 
to the second expression in (3. 43). In particular we shall 
consider six values in the following order; 

SL (j)HSEHJ), corresponding to the paradigm WPEAK; 

5Lj(j)-SH(jh corresponding to the paradigm CPEAK; 
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SLjjJ-SHfjK cor responding to tne paradigm BFEAK; 

SL^j^SHtjh corresponding to the paradigm APEAK; 

SE(j)-5hT(J), corresponding to the edge paradigm of 

Figure 3. &; 

SL_{j)-SH(j), corresponding to the peak paradigm of 

Figure 3. 6- 
In Figures 3. 9 and 3, 10, WC have these Six "'eomporjentti 11 of 
qi## individually graphed above the corresponding Intensity 
profile, in ■the ahove order top to bottom, in a manner simitar 
to that used in Figure 3.7, 

One may make a series of observations concerning the 
various curves in Figures 3, 9 and 3. 10. Comparing the 
lowermost two component corves in the upper left of Figure 3. 9, 
it is easy to see that the pair of peaks in the upper left 
illustration of 3. 7 were doc to the peak -detector profile 
entirely. This follows from the fact that the peaks appear 
only in the lower of the pair of carves from 3. 9 just mentioned, 
Regarding the same profile, one may see that the major peak 
from among the component curves is the uppermost. This 
correbpondE to the profile W'PEAK, which is a. wide peak. 
Evidently the feature represented by the maximum of this 
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curve is the inverted wide peat to the Left of the center of the 
intensity scan- Another observation concerns the lower right 
example in 3. y. rt ie reassuring to note that although the 
discontinuity present in the intensity profile is quite large, 
It was "noticed" by none of the peak -de tec tor profiles. This 
is evidenced by the fact that in the top four component curves, 
and in the bottom one, tne curves are not above threshold in 
the vicinity of the extreme intensity discontinuity, On the 
other hand, the next --to -bottom curve, corresponding to an 
edge, has an extreme peak at this location. Lastly, one may 
make the general observation that tne maximum of all six 
cotnponcnt values of Q 1 ** may be thresholded at a considerably 
higher level thanQ—, for example at a value corresponding 
to aQ'* value of . 99, and still detect all edges, but at a 
considerably better .signal -noise ratio. Observation of the 
figures Shows that at this level a few false positives remain, 
e. g, t the peak to the right of the large discontinuity in the 
lower right example in Figure 3. 9. An examination of the 
intensity curve itself would appear to justify the contention 
that there really was "something there n after all, perhaps 
a spot on the cube, or an insensitive spot on the vi dissector 
photocathode. 
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CHAPTER 4 

A COMPUTER PROGRAM FOR FINDING LINES 



T . 



INTRODUCTION 






Figure 4. 1 
Figure 4. 1 illustrates: the firist stages in the analysis 
of scenes by the program to be described in ibis chapter. On 
the left is a photograph of a set of objects placed beEore a 
random accuse optical input device connected to a digital 
computer. Programs in the computer have access to 
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intensities in the field nf view at arbitrary locations on a 
20000 g by 2 04}OQ s grid. Intensities arc obtained by the program 
along 50 vertical and o0 horizontal scans of 500 intensities each. 
The intensities- are eonverted into ''feature points", as 
illustrated in the center illustration of 4.1. Feature points are 
locations where the scans are adjudged, by spalially local 
processing, to intersect with an edge of an object in the real 
world, Often noise in the real world or within the vidissector 
will produce an intensity configuration along part of a scan 
which is edge-like and gives rise to a "false positive" feature 
point. The right-hflnd portion of Figure 4.1 illustrates the 
output of a program which extracts lines from an array of 
feature points and which is designed to ignore these 
extraneous "false positive" points by detecting chains of 
lined -up feature points. A final Stage of the analysis, not 
illustrated in Figure 4,1, is the proposing and verification 
of lines not located by the above procedure. Lines are proposed 
on ihe basis of lines already located, in places where figures 
appear to be incomplete. 
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XL A DESCRIPTION OF SCENES ANALYZED 

In this chapter we will describe an object recognition 
program baaed on the theory so far developed. This section 
will he devoted to a discussion of the restrictions imposed 
by the theory on the scenes analyzed, and on the optical input 
device used. Various scenes conforming to the restrictions 
will be illustrated. 

The theory developed Id the preceeding chapter e> is 
applicable to certain types of scenes, and to optical input 
devices satisfying certain conditions. Stated informally, the 
restrictions on the scenes are: 

1.1) The scene consists entirely of relatively homogeneous 
regions bprdered by edge lines, 

1.2) The edge lines are all straigbt- 

L 3) The objects of the scene may be recognized entirely 

from the edge lines. 

The first qf these restrictions is fundamental. It is 
virtually equivalent to the assumption that an intensity profile 
across a scene consists of a series of smooth curves bounded 
by discontinuities as in Figure 4.2. 

LIB 



INTENSITY 



HYPOTHETICAL INTENSITY PROFILE ACROSS A 
SCENE IN THE ABSENCE OF BLURRING AND UOlSE 



Figure 4. 2 

The second of these assumptions is essential if one ig 
applying an ed^e 'est only to narrow rectangular regions, as 
is the. cage in the program under c on si deration. Of course 
one could develop a theory, analogous to the foregoing* for 
amalgamating local feature point information over curved bands, 
and thereby detect edge lines of arbitrary shape. However we 
have not done this; it almost certainly would require computation 
of an order of magnitude above that used in the current program. 

The third restriction in a matter o£ convenience. 
Actually the program may be presented with scenes not 
conforming to this restriction without any impairment of its 
behavior. For example, it might be presented with an edge -on 
view of a cylinder, and successfully recognize the outline, 
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The roundness, though deteeLablu by the vidissecior from the 
intensity gradient across the surface, would not be detected by 
the present program. 

These restrictions are summarized in Figure 4,3. 

NOT APPROPRIATE: 
V 





APPROPRIATE j 





ATTENTION WILL BE LIMITED TO A 
CERTAIN CLASS OF SCENES 

Figure 4,3 

The theory requires ■jUtIUlil :^£d-.rki1i::n-rf r>r r'-:e 
instrument which obtains the inter sitiee from the real world- 
These are, in summary* 
2. 1) The ejxrtent to which the input device blurs the image 

received mutt be uniform over th* whole fit Id. 
2.2) The eitent to which the intensities are modulated by 

noise must be unilorm over the field, and independent 



ISO 



of relative intensity. 

Stated more specifically, the first assumption says 
that if Ux,y) represents the intensity of the scene over the 
field of view, D; then the intensity received by the optical 
input device in the absence of noise is: 



Pfoy)" /l(i-u* y-vjflu, v) dudv 

J> 



for Some blurring function f(x, y). 

These assumptions pre summarized in Figure 4, 4; 
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BLURRED INTENSITY 
WITH NOISE 



THE INTENSITIES OBTAINED FROM THE 
OPTICAL INPUT DEVICE ARE UNIFORMLY 
SUBJECTED TO BLURRING AND NOISE 



Figure 4, 4 



Some act lies analyzed by the program are shown in 
Figure 4. 5. 

It is easy to see that criteria 1. 2 and 1- 3 are 
satisfied by scenes of this sort. However, it is not readily 
apparent that criterion L. I is satisfied. This follows from 
the fact that real lines such as highlights and cracks between 
cubes t in the absence of blurring and noise are never of zero 
width as stipulated in the definition of a line- However it is 
easy to feet that they may be of nonzero width so long as they 
arc sufficiently narrow to be indistinguishable from aero -width 
lines under the blurring 1 function. This blurring function was 
discussed in Section II of chapter three. From its properties 
it is easy to see that blurred versions of lines of le££ than 
about five unite width, where one unit is 1/2 000 of the 
yj director field width, are virtually indistinguishable from 
each oLher and from blurred versions of lines of negligible 
width. Criterion l. 1 has thus been satisfied for these figures 
by keeping the apparent widths of the edge lines below the 
aforementioned five units. 

Criterion 2.2 has been met by the optical input device 
used, an Information International vidissector, by the nature 
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EXAMPLES OF SCENES ANALYZED 



Figure 4,5 
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of its design. However certain care is required to satisfy 2. 1, 
and it can not be completely satisfied by the instrument used. 
The blurring inherent in the vidls sector is a function 
of optical focus> of internal electronic focus , and of the size 
of the internal aperture used to collect intensity data from a 
point in the visual field. The latter two matters have been 
discussed at length by Horn {Horn 19&9). and in the literature on 
the instrument itself. Suffice it to say here that there is a 
more or less uniform degree of blurring inherent as a result 
of these two f actor s, though a certain amount of non -uniformity 
is detectable by careful measurements. It is fortunate that the 
order of magnitude of these effects is somewhat greater than the 
optical blurring of the system when it is focused to minimise 
this blurring. By keeping the objects in ihe field of view in a 
sufficiently flat plane, it is possible to hive the non-optical 
blurring effects dominate the optical effects even for portions 
of the field which are relatively out of focus. This, to a first 
order of approximation, insures a uniformity of over -all 
blurring, despite the facr. that the purely optical blurring is 
quite non-uniform. The criteria for "a sufficiently flat plane 11 
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for the image is given by Laws of optics. From the well known 
relation that: 

1/F = l/>+ 1/b 
Where: 

a = object distance 

jj = image distance 

F = focal length of the lens, 
it follows that; 

da/db = - a 2 /b 2 . 
Hence for a given displacement da of the object from perfect 
focus, the image displaces by an amount -EAaHb 2 /a 2 ). This 
quantity, divided by the focal ratio f, gives the amount an image 
is blurred by displacing the object a distance aa from perfect 
focus. Thus the allowable scene depth for a particular 
maximum allowable amount of blur-ring depends un the square 
Of the scene -distance image -distance ratio; and inversely on 
the f -ratio. If a particular object or arrangement of objects 
increases in absolute size, and its distance from the lens 
increases proportionately {and if the object distance is large 
relative to the lens -image distance), its Image remains 
approximately constant in si^e, but the depth of the object 
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remaining in good fo™^ :'n creases linearly reLative to the size 
of the object, It is thus passible, fur a given arrangement of 
objects to keep the blurring over the whole field down below 
any pre -as signed limit by making the f- ratio sufficiently large, 
and by making the objects sufficiently large. In practice, due 
to the low sensitivity of the vidis sector it was often impassible 
to insure adequate depth of field by the use of a suitably small 
aperture alone. In particular ^ in the catie of the objects 
fUustrated in Figure 4. 5, it was necessary to make an effort 
to make the objects as large as possible to satisfy criterion 2. L 



III, THE DETERMINATION OF FEATURE POINTS FROM 

INTENSITY INFORMATION 



In thifi section we will be concerned with a discussion of 
a procedure for extracting information from a set of intensities 
taken over a visual field, as a first step in the determination 
of the locations at lines in the field- The relevant theory, 
developed in the previous chapter, will be reviewed; and. an 
algorithm for reducing intensity information to "feature points' 1 
will be discussed. 

In applying the line predicate to various regions on 
the visual field, we shall obtain intensities along^ scan bines 
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perpendicular Id the axes of the vidissector. In particular we 
shall use intensities obtained from stanfc parallel to the x-axis 
in applying the line predicate to long rectangular regions 
oriented within + 45 e of the y-axis. Similarly, whenever a 

line predicate is applied tu a region within + 45* of the x-axis, 
intensities from scans parallel to the y-axi&. will be used. We 
may thus henceforth without loss of generality assume that we 
arc dealing with :ct;:<;:it: urUwitt-.d wi-rin j_ 4j c of the y-fcLKi£ r and 
with intensities obtained from scans taken parallel to the x-axis, 
A scan sh all consist of 500 intensities taken at intervals of 
2 units along a Line, where, as in the previous chapter, one 
unit is 1/2000 of the full field width. The line is centered 
laterally in the field. Fifty sear.fi will be taken at intervals of 
2 units, with the 25-th scan half way up the field of view. The 
set of scans will thus encompass a square area exactly centered 
in the field of view, whose edge length is 1000 units, or one 
half the edge length of the entire field of view. The reason 
for not using thE entire field is simply that there are various 
physical problems at the edges, one of which involves an 
aperture which prevents obtaining intensity values at the corners, 
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The central square used is approximately the largest useful 
Square region In the field of view. For purpose e of visualizing 
l.if foil owing txpiaoaiitwn, ■'.: may ::* assumed LkuL az: entire 
set of 2500 intensities is obtained at on.ee> prior to any further 
processing. This is not entirely tnje, but may as well be 
considered to be the case. 

We will want to be able to apply the line predicate to 
any of approximately fifteen million rectangular regions in the 
field of view. This figure is obtained from the fact that we 
are considering regions extending between pairs of scans, and 
the latter number &D.*49/2 = 1250; we are considering regions 
at any of fifty orientations; and we are considering regions 
with any of 500 different intercepts In fact we shall apply a 
modified version of tiie predicate to all of a subset of these, 
namely those which traverse the field entirely, which number 
SO* 500 = 25, 000, This will be discussed in Section IV, Also 
the predicate will be applied to shorter subregions of those 
among the latter which appear to contain a line somewhere 
along their lengths. Further the predicate will be applied to 
regions proposed on the basis of lines already found. This will 
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be discussed in. Sectiane v 3 VI and VII. 

The application of Q-* to a region on the field of view 
consists in cumpuLir.g the vaJue given by {3. 13): 

CL(^) + CEU- 1 ) 



CL(jJ) -t CE(J n ) + CH{j") 

where CUJ^) is given by (3. 29}, and CEpf) and CH(j n ) are 
1 J J 

given by similar formulas, A region whose maximum minus 
minimum y -distance is e unite intersects with lis/ 2011a can 
lines, where Hill means the closest integer to L Instead uf 
assuming that a region is of a particular fixed width measured 
perpendicularly to its length, we shall assume it has a fixed 
x- direct ion width of 50 units. Consequently the intensities 
along a particular scan which lie within a rectangular region 
will be exactly 25 in number, since the separation of intensity 
points along a scan is two units. We shall also assume that 
the length of a sub-region, instead of being a fixed quantity 
as measured along the axis of a region, has a fixed y-direction 
length of 20 unite. Also sub ^regions will be taken to be 
vertically centered on a scan line. This arrangement is 
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diagrammed in Figure 4. 6, 



SUB -REGIONS 



SOB -REGION 

SCAN 

SEGMENTS 




SCAN 
LINES 






RECTANGULAR REGION TO 
WHICH PREDICATE IS APPLIED 



Figure 4. 6 

We may thus identify si segment of a scan enclosed within a 
region with a particular Sub -region. The advantages of this 
sort of geometry were discussed at the end of the previous 
Chapter. We shall use a specific terra for the segment of a scan 
which we are associating with a particular sub-region, namely 
a sub -region scan sag merit. In summary, a region of length 
a conEists of lla/20|l = m_ sub -regions; and the value of Q* is 
computed from the rn sub -region scan segments by calculating 
values of CL, CE, and CH according to formulas of the form 
flivL'n hy <3 2 9). 

Evidently the intensities of a particular sub -regies 
scan segment enter into the computation of Q* for several 
thousand regioni= enclosing the segment, Examining (5. 29) 
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it is apparent that, e.g., in computing CL, it is necessary to 
compute a value R from the intensities in this segment, and 
also a value- a? : and that CL is determined entirely from these 
values. It is not hard to see that these values are the same for 
a particular segment when the enclosing regions are all at the 
same inclination. Further, it was pointed out in some detail 
in the last chapter that these values for a particular segment 
do not vary a great deal with orientation of the enclosing 
rectangular region assuming, as we are, that the regions 
remain within + 45* cf a normal to the scan lines. Thus, as 
before, only values of R etc. t for two region orientations, 

LD°and 30° , need be computed- This is dene, as In the previous 
chapter, by using two paradigm profiles of different widths, 
Finally, we may recall that at the end of the previous chapter 
the procedure of computing a single value of Ft, R 1 . etc. , for 
a particular segment and orientation was generalized to the 
computation of several, corresponding to different possibilities 
of the shape of the intensity profiles toward the edge of the 
region. Consequently, for a given sub -region scan aegment, 
we will compute exactly the values K. . -> . ,. , R . 

iai 
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a'* , And also H" arid a 11 * , The six R's and a*'s are 
j.k, 5 j,k J,k 

computed as in {3. 25) and (3, 23) from six different peaked 
profiles (U-, ... U, }, r . . , (U^, ... , U^ ) of various widths 
and with various combinations of gradients towards the ends, 
Similarly the R ni s and a'*'s are computed from five different. 
Cliff -like profiles- Clearly, no tuch multiplicity is necessary 
In the case of R 11 and a r '* J which corresponds to a homogeneous 

region. These paradigm profiles, £5- tuples of values (U„ 

25 
. r , , U, }. etc, from which the K's, a*'Sj R"a t a'*'s, and 

R" and a"* are compute d p are Illustrated in Figure 4. l r 

Since values of R, . ,, a* .,---, R 1 . , r , a'* 

2iKl ],k, 1 J,k, 5 j,k;, 5 

R" , i and a'. 1 * will be required for every sub-region Scan 
', . ^ Jj k 

segment, they are all computed- Since there are 475*50 
sub -region scan segments, and 24 values computed per segment, 
this is a matter of 570,000 intermediate values t an order of 
magnitude more than the 25-, (J GO intensity values from which 
the values were computed. However, the derived set of 
values admits of considerable compactifieation. P'iret, for 
a particular sub-region scan segment it is necessary to retain 
only the minimum of all the R's and R M s, Logether with the 
corresponding value of a* or a 1 *. Secondly, consider the 
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PARADIGM INTENSITY PROFILES USED IN COMPUTING Q 1 ** 

FigurB 4.7 
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values of R" minus this minimal R p-r R 1 for the 475 successive 
sub -region scan segments along a particular scan. Then there 
exists a threshold for these differences such that most values 
lie below the threshold and indicate with virtual certainty that 
aline does not traverse a region containing that segment. 
Such values may be omitted, thereby indicating that they fall 
within this category, AlsOj only the local maxima of the value*;, 
above threshold need be considered- Consequently an entire 
collection of 475 sets of values may be reduced to Somewhere 
around ten seta. These latter sets need only consist of 4 
values: the value of R" minus the minimal R or R 1 which is 
locally maximal? the value of a* or a 1 * eorreipnnding to the 
minimal value af R Or R'; an indicator as to which R. Or H 1 
was minimal; and the x co-ordinate of the point in question- 
Such a four -tuple, together with the y co-ordinate of the scan 
from which it was derived, will be termed a feature point, 
We will not justify in detail here why the 570, 000 values can 
be reduced to approximately 500 feature points. However sve 
shall point out that the subsequent use made of derived 
information In the computation of Q* and. similar values uses 
no more than the reduced feature point data; that many of the 
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values can be considered to be in a single null category; end 
that the remaining values nave fcuff icient regularity to be 
specifiable in terms of their local majniuia only. 

The program {GETFP X S) obtains intensitlM for a 
single scan and performs the calculations sketched above. 
The second argument is 1 or depending cm whether a scan is 
to be made vertically or horizontally respectively. The value 
of X is the appropriate ordinate or abcissa on a scale frem 
to 500, Five hundred intensities are taken across the field 
along the line defined by the arguments and are processed 
according to the algorithm sketched above. The output is a 
series of 4 -tuples as described above. A eehematic diagram 
of this process appears in Figure 4, 8, 

In the line finding 1 program which has been developed, 
:\-.v. Hi:b program GETFP itf ::n!U-rL "■ "i tlries for ■-■ u r I : <j al Scans 
and SO times for horizontal scans. This procedure is illustrated 
in. figure 4. 9. tn this figure, the short lines indicate the 
locations of maxima previously described. Their lengths are 
proportion al to the corresponding values of R ir - Min(R, R r ) 
for small values of this difference, for values of this difference 
above a certain threshold, a constant length of line is used 
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corresponding to this threshold value. These points will be 
termed feature points. Rasters of feature points corresponding 
to the illustrations in Figure 4. 5 are given in Figure 4. 10, 



IV* A GLOBAL PROCEDURE FOR EXTRACTING LINES 
FROM ARRAYS OF FEATURE POINTS 



If orte were 1o ex amine the feature point rasters of 
Figure 4. 10, One would be able to pick out almost all the lines 
In the figures they represent. The perception of some of the 
lines may depend on complex global perception processes, 
for example, the use of obvious lines lo provide clues &s to 
the existence of less obvious ones. A computer approximation 
to this process will be the subject of subsequent sections . On 
the other hand, some of the lines are obvious without either 
reference to other parts of the figure or a priori knowledge 
of what constitutes a plausible figure. This is because the feature 
points are both strong (represented in 4.L0 by long lines \ and 
they line up very exactly. This section describes a program., 
LTNES, which takes advantage of the obviousness of certain 
lines to find them in an array of feature points, by a rather 
simple and not overly lengthy procedure. 
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FEATURE POINTS EXTRACTED FROM THE SCENES IN FIG, 4,5 

Figure 4.10 
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The problem of locating lines by accumulating locally 
obtained information has been approached by line following 
procedures. £ Roberts) Thi* often runs into the problem that 
the line follower gets lost, since, in effect, it is making a 
series of decisions (whether or not to continue the line) on 
the basis of local considerations. This problem becomes 
greater as the number of lines In the scene increases. Line 
followers are also subject to the problem o£ never having 
"attached" them selves to certain lines in :he first place. It 
is thus desirable to use a more exhaustive and global procedure, 
lor example, covering the feature point raster with a very large 
number of narrow rectangles, and applying a thresholded 
predicate to the feature points within each rectangle, The 
apparent tremendous cost of such a procedure can be greatly 
reduced by observing that if a certain narrow rectangle contains 
evidence of a line, then so does a rectangle with the same 
orientation but extending across the entire field. The converse 
of this statement Is not true, since a rectangle across the 
whole field may contain a large amount of line evidence in "the 
form of scattered noise and spurious values due to feature points 
along lines intersecting the rectangular region at some angle. 
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However, the testing of narrow rectangles across the whole field 
does provide a sort of screening procedure. Any relatively 
strong lines will fall within at least one of these bands , and thus 
he detected. On the other hand, it is necessary to further analyst e 
1he contents of one of these hands which has a large amount of line 
evidence in it in order to determine both if there really is a 
Line within the band; and if so, where it is. 

The procedure for assessing the total amount of evidence 
for the existence of a line within a band is based on the idea of 
projecting a two dimensional array lnxo a ojlc dimensional line. 
For example, if an array of points (x^ y.) is given, and it la 
desired to determine the locations {x -intercepts) of vertical 
bands containing a large number of points, then it is roe rely 
necessary to histogram the x- co-ordinates of all the points. 
The resulting histogram has large values for x-intercept 
values of bands containing a large number of points. By this 
procedure, all vertical hands are investigated in parallel, 
The procedure is simply modified for arbitrary orientations, 
hy bistograming numbers of point? versus a quantity 
x i +■ ay where a depends on the orientation under consideration. 
This corresponds to projecting the raster of feature points onto 
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the x-axis at some angle, 

The program (FROJ X SL> carries out this projection 
procedure on ar, array of feature points X, projecting at an 
angle SL. The main program calling PRO,! uses 100 valued of 
SL. For &0 values, the feature points obtained from horizontal 
scans (parallel to the x-sxis) are used- The values of SL in this 
case correspond to orientations between ± 45 degrees wilh the 
vertical (the y-axis). The remaining 50 projections use the 
feature points irom the vertical scans, and UwoWe angles 
between i 45 degrees with the horizontal- The value* histogramed 
zr* not simply numbers of points, but fiimis of integer weights 
attached to the various feature points. These integer* have 
values from 1 to 4. and depend monotonically on the value of 
K 11 - MintR, K'h the firal feature point value, for a particular 
point. Thus, for a particular band, it is not simply the quantity 
of feature points within it that is considered, but something 
approximately equal to the sum of the values of R" - Min<R, ■') 
for points within it. This gives extra weight to distinct Unas, 
and may be shown to approximate Q* in some sense. The value 
returned by (FftOJ X SL) is the set of ^-intercepts of centerings 
of bands of slope Si, which have a large amount Q £ line evidence 
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in them. Til* algorithm employed by PROJ is diagrammed in 
Figure 4. 11. 

The set of outputs of (PRO J X SL) must be analyzed to 
extract the actual lines in the scene. One problem which must 
be taken into account arises from the fact that a particular set 
of feature points may give rise to an intercept value in the 
output of several applications of PRO J. for successive values 
of SL. This is a result of the fact that a given set of feature 
points which fall in aline along the visual field are contained 
within sever al rectangleB of almost identical a lope s. Another 
problem is that the output Of PROJ contains a large number of 
false -positive values due to spurious effects described 

previously. 

The program LINES, which calls PROJ for various 
values of the arguments , takes these factors into account by a 
complex process of: 
1> retrieving the feature points hi sets from areas 

E.L-j'^FiFiTeir: by ihc 0,iTpi;t a: * 3 HO.r, 

2) eliminating sets which dn not actually constitute lines, 

3) eliminating spurious points from sets which do represent 
lines, 
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SIMPLIFIED DIAGRAM OF {PRO J X SL] APPLIED TO 

X, A SIMULATED FEATURE POINT RASTER, AND 

SL, A SLOPE CORRESPONDING TO <p IN THE DIAGRAM, 



Figure- 4.1.1. 
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4) sorting by the apparent strength 0/ linea, 

5) eliminating redundant sets of points by considering the 
set in the order imposed by step 4 and eliminating sets 
which are highly redundant with (more distinct) sets 
which had previously been considered, 

*:■) fitting lines to the resulting sets pf pointy 

The output of (LINES 20 is a set of lines retrieved from the set 
of feature points X. LINES is applied once to the feature points 
from the vertical scaji; and once to the feature points from the 
horizontal scan. The result of applying LINES to the sets of 
feature points illustrated in Figure 4. 10 is shown in Figure 4. 12. 

V. LINKING LINES TO FORM PARTIAL FIGURES 

Having located a certain number of lines in a scene by 
the methods of the previous section, a logical next step is to 
join them together and form figures. This may be viewed as a 
terminal step, provided all lines have been located by the 
projection procedure. This would not, in general, be the case; 
and the joining procedure descrjhed in this section is actually 
a first step in the location of the remaining lines of the figure. 
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OUTPUT OF LINES, A F PLIED TO THE FEATURE POINT 
RASTERS IN FIGURE 4. 10 

Figure 4.12. 
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We shall assume that the output of LINES, when applied 
to a Set of feature points, does not include all the lines of 

interest in the original scene r This is, In fact, the naae for 3 
out of 4 scenes in Figure 4. 12. It is a reasonable hypothesis 
that the lines found, together with some assumptions about I he 
nature of the scenes examined, provide some clues aa to the 
locations of the remaining lines. It appears not 10 be the lines 
individually which suggest the locations of other lines, but 
rather collections of line e which form a figure which is 
incomplete relative to a model of how the actual figures appear r 
An example is Shown in Figure 4. 13, 




THE SOLID LINES HAVE BEEN IDENTIFIED, 
THE DOTTED LINE IS SUGGESTED BY THEM. 

Figure 4. .13 
It would be desirable to join the lines mLo groups which would 
suggest the possible location of other lines, Aline verifier 
could then be applied to the locations suggested. This 
procedure requires that all locations of missing lines ultimately 
he proposed. Otherwise,, some lines would never be found by 
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either procedure- Lt is thus important that the whole ipeehaniem 
for line propoeiMj b* liberal in proposing lines- Any excess lines 
proposed will almost certainly be rejected hy a suitably powerful 

verifier. 

prior to the application of the linking procedure, a 
merging procedure is applied to the lines. This is necessary 
to eliminate redundant lines from two sourceet 
i) Lines which are at approximately at a 45-degree angle 

with the vertical, and appear in :he output ol LINES 

applied 10 both the vertical and the horizontal scans- 
2) Single lines which appear, for one reason or another, 

as a pair of contiguous lines in the output of LI«ES. 

The criterion for merging lines is given in Figure 4, 14, 
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|dl+le|+1d+*|<50 

CRITERIA FOR MERGING LINES 

Figure 4. 14 
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XI two lines are to be merged by this criterion, the pair is 

replaced by a single line which best fits the endpoints of the 

two original lines. 

The joining procedure consists in applying directed 

links from one line to another. There are four possible types; 

1) Regular links, denoted by the symbol T, If any part of 

line B lies within a neighborhood of radius 10 units 
around A, then A is given a directed T-link to B, 
This is illustrated in Figure 4. 1&. 

2} Extension links, denoted by the symbol E. II any part 

of line B lies in a particular region n#ar the end of A. 
or if the end of line B lies within another more distant 
region extending away Iram the end of line Aj then an 
E-link from A to B is established. These regions are 
illustrated in Figure 4- 15- The motivation for this type 
of link is that possibly a segment of A extending to B, or 
a longer segment extending to the end of B p has been 
omitted. The reason lor postulating a rather long 
extension to the end of another line is that a line's 
endj lining up with s given Line, provides strong- 
evidence that the original line may actually extend lo 
that end, 
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A LINK IS ESTABLISHED FROM LINE A 

TO LINE B ON THE: BASIS OF THE INTERSECTION 

OF B WITH CERTAIN REGIONS AROUND A 



Figure 4. 15 

3) Passive regular links, denoted by FT. If line B links 
to line A by a T-link, but A docs not link to B by either 
type of link* then a FT -link is established from A to B- 

4 ) Pssgive extension Link, denoted by PE- This is similar 
to a. PT-link, but is applied if B links to A by an E-link. 
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Ail illustration of the links applied to one of the sets of 
lines from Figure 4. 12 is given in Figure 4, 16. It should be 
emphasized thai, in accordance with the necessary liberal 
policy underlying the proposing of lines, the lines are over -linked. 

These links are not intended to be those applied when all the lines 
have btcn lueated. 

VI. HEURISTIC LINE PROPOSING 

This section describes the program (PROPOS X}, which 
proposes additional lines from a possibly incomplete set X of 
lines from a scene. Each line is considered in turn, and 
additional lines suggested, by it, and by lines linked to it, are 
proposed. A verification program, to be explained in the last 
section, is applied to each prnpofied line. If the verifier claims 
that the line actually exists, it is added to ihe list of Lines, 
together with links between it and other lines; and it may be 
used subsequently in proposing further lines. When no more 
lines can be proposed from the (possibly augmented) 1 set of lines, 
FKOPQS returns the entire set. 

The proposing procedure is based on the premise- ihat if 
a line is missing from a set S, then at least one hue having a 
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LINKS BETWEEN MEMBERS OF THE OUTPUT. OF LINES 



Figure 4. 16 
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com man end -point with the missing line is in the- set X, or can 

be successfully propo-ted from X. This is illustrated in 
Figure 4. L?. ' It is further assumed that vertices have no more 
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IT IS ASSUMED THAT IF A LINE OF THE 
ORIGINAL FIGURE WAS NOT AMONG THOSE 
OUTPUT BY LINES, THEN AT LEAST ONE 
LINE CONNECTING TO IT WAS t 

Figure 4. 17 

than three or possibly four ljnts radiating from them. It 

follows that lines should be proposed from the ends of lines 

already found, provided that there are not already many lines 

incident with a particular end. 

It Is in general not entirely cltar where a. line- generated 

by LINES actually terminates. This is partially due to the 

rough "mean 11 of the scanning gridL and partly due to the fact 

that the feature point generator gets confused in the neighborhood 

of a vertex. In order to propose lines from thE ends of known 
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lines, it is necessary to make a reasonable estimate as to 
exactly where the ends lie. The first step in proposing lines 
from a given line is to dispose of the ends in the following 



manner- 



1) If two lines intersect within ten units or the end of a given 
line, the point on the line (possibly extended) closest to 
the intersection is t alien as the end or "terminus". 

2) If a terminus is not determinable by criterion 1, but a 
line intersects the given line within 2 5 units of the end. 
this intersection is taken as the terminus. 

3) If no terminus can be found by 1 or 2, that end of the 
line is said to be of "indeterminate terminus, " 

The determination of termini is diagrammed in Figure 4, l£ r 

Another assumption made by FROPOS, which is 
independent of the first, is that the opposite sides of faces of 
trie figures are approximately parallel. If it is desired to 
propose a Line extending^ e, g. , downward from the left 
terminus of a horizontal line^ the direction of the proposed 
line is then the same as that of another "secondary 11 line 
extending downward from the given line. In particular, this 
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1 'x. terminui 

THE "TERMINr" OF THE HORIZONTAL 
LINES AHK AS INDICATE ]>. 
LINES PROPOSED FROM THE HORIZONTAL 
LINES EXTEND FROM THE TERMINI. 



Figure 4, 13 
"secondary" line shouLd be the first line "to the right" of the 
Left terminus, as that is the obvious candidate for an opposite 
side to the proposed one. This deter mj nation of these secondary 
lines, termed "directors 11 £ B diagrajnmed ha Figure 4. 1!J. 
Having determined the termini and directors for a 
particular Line, the procedure for proposing lines from a 
particular end is governed by four cases: 
l ) The end oi the line is of undetermined terminus; 
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UPPER DIREC1 
FOR LEFT END OF A 



LIME "A"-* 
LOWER DIRECTOR** 
FOR LEFT END OF 
LINE A 




NOT DIRECTORS 

FOR LEFT END 

OF LINE A 



DIRECTORS ARE LINES WHOSE DIRECTION 
SUGGESTS THE DIRECTION OF A MISSING LINE. 



Figure 4, LB 

2) Only one line intersects with the given Line at the 
selected end; 

3) Two lines lying on the same side of the given line 
intersect the selected end; or 

4) Something eLse occurs at the selected end. 

Ln the first case lines are proposed from either side of 
the original line for which a director has been found. Three 
parallel lines are proposed, ID units apart, parallel to the 
director, and with Uie end of the middle Line coinciding with 
the indicated end of the given line. The other ends of the 
proposed lines are determined in this case, and in the cases 
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which follow, by making the proposed line no longer than the 
director, and truncating it so that it does not intersect any othei 
lines in the set. In CM e 2h lines are proposed on the opposite 
side from the intersecting Line, provided a director exists on 
that side. Three lines are proposed ten degrees apart, all 
extending from the terminus. Their lengths are determined 
&e above, and the middle line is again parallel to the director. 
If no lines are found on the aide away from the intersecting line, 
then lines are similarly proposed on the same side as the 
intersecting line, subject to the additional provision that the 
proposed lines all lie between the intersecting Line and the 
director. If this were not the case, it would be impossible that 
the proposed line and the director be opposite sides of a face. 
]n case 3h lines are proposed similarly to the first set 
described for case 2). In case 4h no lines are proposed. 
These cases are diagrammed in Figure 4,20, 

In summary, PROPOS goes down the list of linked lines 
proposing in the manner described above, adding lines which it 
finds to the end of the List. This procedure Ls applied to the 
resulting list, and so on untiL no lines on the current list result 
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1) LOWER TERMINUS OF "A" IS UNDEFINED, 

PARALLEL LINES PROPOSED: 

DIRECTORS FOR 

LOWER END OF "A' 1 
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2) ONE LINE INTERSECTS LOWER END OF "A", 
RADIAL LINES PROPOSED: 




LINE "A 11 
LINE "b" 




o TERMINUS 

PROPOSED 

LINES 



% PROPOSED LINE TRUNCATED BECAUSE OF "B" 

3) TWO LOSES AT LOWER END OF "A" ON ONE SrDE, 
RADIAL LINES PROPOSED ON OTHER SIDE' 

o TERMJNUS 

PROPOSED 

LIKES 




4) LINES ON EITHER SIDE OF LOWER END OF "A", 
NO LINES PROPOSED: 




PROPOSING OF LINES FROM AN END OF A GIVEN ONE 
IS GOVERNED BY FOUR CASES. 

Figure 4.20 
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in the proposing of any further linos- At this pointy the final 
flet df lints i£ returned by the program. The process of 
proposing lines is diagrammed In Figure 4.2L, 

VII. A LINE VERIFIER 

Line veriii.cat.ion consists in applying the function 

Q*(Jj) as given by (3 r L3) and (3, 29), or some variant of it, to 

n 

L.'lc i n L «.- r j 1= i ti c:- 1= .1 ur ^::ou:- v^Lilc-^ derived \'r:y.ii them, lying 

in a region suggested by the line proposer. The region is 
assumed to be divided into n Square or nearly Square eub- 
regions, andQ* or its variant is computed from the values 
R ... , R t a* . ... , etc., according to Formulas 

(3. 13} and 43, 29), or formulas derived from them. In most of 
the remainder nf thia section we shall be concerned with a 
generalization of Q-'-tJj ) which has been suggested by the 
discussion at the end of chapter three. This generalisation 
consists in assuming that the values uf R- . and E! are 
given by; 

K l,k = Mi "«U,, "Us 1 - (42) 
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SMALL NUMBERS INDICATE THE ORDER OF PROPOSING. 



Figure 4.21 
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where the R, . Vb and R! , 'a are computed from the 25 

intensities of the k-th sub-region scam segment according to 

(3. 25). and from the paradigm intensity profiles illustrated in 

Figure 4, 7- According to the discussion in Section III, the 

values in the feature point rasters are derived from the 

values given by (4, 1) and (4, 2), and the present line verifier 

draws upon exactly these values. In this section we shall 

review the relevant theory in detail, describe the present line 

verifier, and suggest possible improvements and generalisations 

of it. 

The present line verifier, in computing a variant of 

Q*(J^) for a region, obtains the values of, e, g- t H for 

CL(j) of [3, 29) from the feature point rasters such as are 

partially depicted in Figure- 4. 10. In principle, if the region 

encloses n 25 point sub -region scan segments of the original 

scans, then the values R. , etc. , should be computed from 

]■> -K 

exactly these scan segments. These values were computed 
for all possible sub-region scan segments, in the generalised 
manner given by (4. 1) and {*- Zh by the feature point generation 
procedure. However, most of the values of the R's and R" ' S 



were discarded during the compact i lie ation process. In fact it 

was only locally maximal values erf E, given by; 

E, , = R ,H , - Min(E. t >R: J (4,3) 

where R and: H 1 are given by (4. 1) and (4, 2), that were 

retained. However. It turn* put that we may obtain a 

reasonable estimate of the value of E. , for the intensities of 

]. k 

a particular sub -region scan segment along a particular scan 
from one of these maxima nearby along the same scan. In 
particular, if E- . is a desired value of E for a sub -region 

scan segment alontf soi^.e ac.an r then let E be the value of 
E which is locally maximal along the same scan and is computed 
from a sub -region scan segment closest to ihe one in question. 
We assume that there exists a maximum within something lite 
15 units of the segment in question, else we assign E the 

A 

value 0. If E. .so defined actually exists, then the following 
relationship holds; 

E *= E - ax (4, 4) 

where a_ is snme constant, and x is the distance hetween the 

A 

Bub- region scan segments from which E. , and E. were 
computed. This relationship follows from the behavior of the 
E curve in the neighborhood of a maximum, which maybe 
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observed in Figures 3 r 9 and 3. Lth It will be seen later that 
the n values of E. , are all thai are necessary to compute the- 
present variant of Q* used in the line verifier. 

This present variant of Q* omits the sfi^ond and third 

terms in the exponent of, e r g, , the value of CL as given by 

(3. 29), and thus does not need the values of a* , a 1 * and 

h Is h^ 

aV*. . The middle term may be ignored because it turns out to 
ji k 

be always very small relative to the other two. To see this, 
recall that line verification in the present set oi programs is 
applied only to lines, or possible Lines, which escaped 
detection by the LINES program. They are thus lines which 
have a net relative amplitude whose magnitude is small relative 
to the average magnitude of relative amplitudes for lines in 
general. One may observe that the denominator of the middle 

term is dominated by its third term, so the term Is approximately 

9 2 

! Jay Jn]/p . This Is approximately the sauare of the net 
k h K i 

amplitude divided by its variance. Since the net amplitude 
in situations under consideration is small relative to the 
average magnitude of the net amplitudes, and the average net 
amplitude over lines in general is usero; then the net amplitude 
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is small relative to lte standard deviation. The second term of 
(3 + 29) is thus in this case very smell, being approximately the 
square oi the quotient of the net amplitude with tb* standard 

deviation. 

The omission of the third term of the exponent of (3. 39) 
W as based on the empirical abservation that for the majority 
of the scenes analyzed, this value seemed not to he related in 
a reliable way to the existence of Lines. A series of experiments 
was begun to properly test whether any such relation existed. 
The results appeared promising- but not so far conclusive. 
Consequently the third term was omitted pending further 
investigation. 

Ths resulting variant of Q*(Jj) is quite similar lo the one 
discussed at the end of the previous chapter. It is given by (3, 13}, 
where the various values, e, g- , of CL. are given by (3. 29) but 
with only the first term in the exponent. This was the case in 
the situation described in the end of chapter three. Alan, this 
time we expect a scan of 500 points to intersect with abou* five 
lines in the visual field, so the relation between F<CU. Fl.CE) 
and P(CHJ remains approximately the same as in the situation 
at the end oi chapter three. Finally, we may as before 
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assume that one of the terms in the numerator is considerably 
larger than the other. Thus by an argument similar to that 
used in deriving (3. 41) and {3. 43) ive arrive At the function 
Q'**4J7) defined by: 



Q r **Uj) 



n 

^ M ax( S L k ( j ) -SH < j >, SE k ( j )-SH( j H + K, (4.5) 

it ~ 1 



I 

where, e. g, , SL^ij) is like SL(j) given by (3. 3?) with subscript 
k in plate of 1, As before, this function is approximately 
monotone inQ*(J:) and hence thresholding it provides an 
approximately sub -optimal regional decision predicate, One 
may see from (3. 3 7), {3. 38), (3. 39). (3.23), (3.24), (3.25) 
and a certain amount of algebra that: 

and 

"V> - -&* ' "1.1. - B j,k- 

Using (4r 3) we may rewrite (4, 5) as: 

n 

■Q'**(j") = T>: t + K, (4.6) 

with the values of E. . obtainable via (4, 4} from values 

A 

E. , , which are available i:i the feature point raster. In 
j,k' r 
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summary we have: 



q^jMe^-4^, C« 



=iV 



recalling that If & suitably close feature point for obtaining a 
value E. ta did not exist, then the corresponding E^ . was 
assigned the value zero. 

In practice it was found to give better- results to 
deviate slightly from a strict thresholding of<4. 7}in the 
verification of lines. Prior to applying (4- 7) to a region it was 
observed which of the E . , were given nonzero values. In the 
case that the locations of the nonzero E 'b were sparse or 
concentrated in one place in the region, the thresholding 
procedure was omitted, and a negative answer- was returned 
as to the existence of a lint. This is approximately the same 
m observing the pattern of the locations of feature points within 
the region, and denying the existence of a line if they are all 
clumped together or are sparsely scattered within the region. 
these two situation are usually due to the existence of feature 
points from lines that cross the region at non -negligible angles 
Another variation from (4- 7} which was found to be more 
convenient, and in same cases more accurate, was to 
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threshold the sums of the E^ .'b and the sums of the x, 's 
separately. Finally, instead of taking x.^ to he the deviation of 
the E. .feature point from the center of the region, it was taken 
to be the deviation of the E feature point from a best -fit line 

A 

to the various E feature; points. 

The line verifier (VERIFY PI PZ> takes as arguments 
the end -points PI and P2 of the proposed lint and ou'.putii 
either NIL, if the line is thought not to exist by the above 
criterion; or the end -points of the best fit line to the feature 
points il the line is considered to exist. 

In observing the line verifier in action in Figure 4. 21, 
occasional failures are apparent. Although no lines are claimed 
to exist which do not exist in the figure, several actually 
existing in the figure were proposed bul claimed, on the basis 
of the feature point raster information, not to exist by the 
verifier, An examination of the feature point raster indicates 
the presence of feature points in these regions. The problem 
was principally that the region intersected with too few fitans, on 
the order of three to five; giving too few values for the threshold 
function ^4. 7) to work with. Evidently a fintr Scan would 
chi::i[jb=.;e ::\'.L iHO'-len, n^ wf I'i .-jH son r- nth fir nporoacirfifl tn 

167 



its improvement which are abou: to be disuussed. 

A major source of improvement for the line verifier is 
for it to obtain a new Set of intensities within the region to which 
It is applied, at considerably higher density then that of the 
original scan- This would eliminate the difficulty described in 
the previous paragraph, would allow more accurate values of 
the R'b, E'b or whatever, to be obtained, and would eliminate 
the necessity of the approximation to the E ■ ^s in {4, 4), 
Another improvement would result from the utilization of the 
third term of (3, 29)- and similar expressions- As was Stated 
previously, investigations in this area are still under way. 
It seems plausible that the model in Section IV of chapter thr« 
may have to be modified In assumptions 4) and S) to admit of a 
regular linear variation of "idealized 11 relative amplitude along 
a noise-free ridge-like or cliff-like noise-fre* sample. 
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CHAPTER 5 

RELATED WORK 

I. OTHER APPROACHES TO LINE FINDING 

An early line -finding prugrarr. was written by Roberts 
(Roberts), This program analyzed photography of good 
contract And focus, of scenes whose content was principally 
defined by straight edge -lines, Feature paints were obtained 
from a predicate on four adjacent intensities of a 2 by 2 square 
on the visual field. Feature points were organized into lines 
by a sort of line -following procedure. Higher level heuristics 
handled the resulting lines, 

A comparison may he made between the behavior of 
Roberts' four -point feature point predicate, and the present 
feature point gathering procedure. This analysis is a 
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generalization of a procedure due to Biniord and rterskovits 

{HerBkovLts), 

An upper hound on the performance of the former may 
be inferred from theoretical considerations. Data on the actual 
performance is not available. The feature point predicate is 
applied to intensities x. ^ z.^., ^ - +1 * and *i + i r ^i at fc,,ir 
adjacent points in a square pattern ona25$b y 25& point grid- 

The threfiholded value is; 



Where 



y i,r^j- 

Assume that the noise is gaussian, and thai the noise level, as 
measured by the standard error about a mean of values of y_ 
taken at a single point under constant lighting conditions, is s 
far intensities in the vicinity of the average of the four y>s in 
question. It is not difficult to show that z as a function of the 

random variables y t ., y i+1 j> *I. j+l and y i+l, j+i haB * 
median of about 2s, and an upper interquartile point value of 
about 3e In the context of the set of figures we are considering, 

one would expect about one feature point per 100 quadruples of 
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intensity. Thus fur about 50% false positives, one would wish 
to set the threshold on ? high enough that z falls above threshold 
by chance once in, 1QD times. Fop a normal distribution, this is 
about 2- 5 standard deviations, which corresponds to 3 times the 
semi -interquartile range. Thus s should cut off at about 5s. 
Now it is not difficult tn sec whai values of z_ would result for 
various lines and edges of amplitude a oriented in various ways 
with respect to the grid of squares. In Figure 5, 1 we have 
computed such values for 12 cases. In the diagram, we are 
assuming that thfi dimensions of tbt blurring function art Such 
as to give the line and edge profiles of the width shown, This 
width appears to be about an optimum oom promise,, if 
considerably lower then more- edj-es would have a relatively 
high s -value, but more lines would have a zero z- value. As 
the width increases, both lines and edges tend toward having a 
zero z-value- From the diagram it may be estimated that the 
average value of z. for edges is about r Ba p and for lines about - 4a- 
Thus the amplitudes of lines and edges respectively which are at 
threshold are about G. 3s and 12. 6s. This may be compared 
with the values 1. 2s and 2s obtained for Q* at the end of 
chapter three. The difference is due to two factors. First, 
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the Roberts predicate has a support of only four points, whereas 
the Q* tested had a support of 25 points. One would expect the 
signal to noise ratio* to differ by a factor ofyf^H = 2.5. 
The actual difference is about 5, and the remaining factor of 
two is probably due to the fairt that the Roberta predicate is 
isotropic, whereas Q* ia directional. 

The Topolo^isL, based on the work at Binford, Subs man 
■(Sussman) and Herskovr;s (Herskovite), represents another 
approach :o the analysis of simple scenes. In this case, the 
scene is divided into a set of small square regions which are 
individually tested for homogeneity. Adjacent homogeneous 
regions are clustered into maximal homogeneous regions, which 
are presumably the areas of the scene which lie between edge 
lines. This procedure has the properties that edge lines need 
not he straight, and that arbitrarily complex predicates for 
homogeneity may be used. 

Recently, a method of spatial frequency filtering and 
matching has been applied to the problem of location of feature 
points {Hueckelh This procedure is roughly a two dimensional 
version of the feature point analysis presented here. For a 
circular neighbor hand, an estimate is given of the directional 
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and amplitude parameter a of a pOBsible edge Through the 
neighborhood. This procedure £eems to be bated en a rather 
simple model of the nature of intensity prof ilea at edges. Also, 
it seams to be relatively insensitive to the presence of lines. 

An approach bearing some mathematical similarities to 
the feature point detection procedure reported here, wag 
employed by Saunders (Saunders) in an analysis of filmed 
images of scope tracings. In this case it was necessary to 
Locate the intersections with parallel scan lines of the image of 
a narrow trace along the film, 

Worthy Of mention here, though not specifically related 
to line finding, is the work in statistical decision theory of 
Chow (Chow). The theorem in the fir&t chapter is essentially 
an adaptation and generalization of Chow's results in the theory 
of optimal error reject trade-off. 



II. RELATION OF THE PRESENT WORK TO THE 

DEVELOPMENT OF A VISUALLY ORIENTED REAL- 
TIME OBJECT MANIPULATOR 



The program here described is part of a larger effort to 
devirlop the visual -perception aspects of a real-time object 
manipulator. (MAC Progress Report 1868) For the present. 
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pft'ort has centered on programs to obtain descriptions of scenes 
composed of cubes and other prismatic solids. By a description 
of an object is meant a list of the spatial co-ordinates of its 
vertices, together with a list of pairs of vertices defining edges 
of the object- By a scene flies criptiwi is meant a Ust of object 
descriptions, one fojf each object in the scene.. 

There Is an obvious disparity between the output of the 
present program and a corresponding scene description. The 
former is, or can easily be reduced to,, a set of vertices and a 
set of vertex pairs representing- edge lines, A description 
requires that these vertices and vertex pairs be grouped into 
sets corresponding to the various objects in the scene. One 
may regard the output of the current program aa a aet of lines 
dividing the field into regions. The problem then becomes one 
of grouping regions which correspond to faces of a single object, 
and matching regions which may be parts pf a single face which 
has an occluding object In front of it which divides it into two 
regions, This problem has been investigated in detail, and a 
program called SEE (Guzman) performs the described gremping 
and matching to reduce the output oi the present program into a 
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proper two dimensional Scene description. 

Several approaches to the problem of determining the 
La cations of objects m three dimensions have been investigated 
in greater or lesser degrees of detail. One suggestion involves 
*he u^e nf two light sources located close to the point of view of 
the optical input device. One would be a point source, whose 
intensity falls off as the square of the distance from the point of 
view. The other would be a source placed "at infinity" by me any 
of Lenses, whose intensity would fall off negligibly with Intensity. 
If illumination is rapidly alternated between these two sources, 
and intensities obtained tor a particular point under both 
conditions, the relation between the two intensities would yield 
the distance from the point of view to the point in the field. 
Another approach to determining the distance from the point of 
view to a pnjnt in the scene involves the use of optical focusing. 
(Horn 1968) By this procedure, a point in the scene is brought 
into perfect focus and the position of the lens of the optical input 
device is transmitted to the computer. After a suitable 
calibration procedure, the absoLute distances of objects may thus 
be obtained automatically- A third approach (Horn) involves the 
use of intensity gradient to infer the shape of the contour of a 
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curved surface- By this technique, reLative distance informal ion 
may be obtained for points oa curved surfaces. A filial approach 
worked out by the present author t involving an analysis of the 
scene from two different points of view, relates directly to the 
current wort*. It is assumed that the present program has 
examined a suitable scene from two different points in space, 
and that the resulting two sets of lines have been processed by 
SEE, Central tn this approach is a subroutine called WHERE 
which is baaed on results in projective geometry, and is 
described in a recent Project MAC Artificial Intelligence Memo 
{Minsky). This program takes as input the two dimensiunal 
co-ordinates of a point from each scene. In many cases, WHERE 
can determine that the two points are not images of the same 
point in space. If the points are from the aame point in space, 
WHERE outputs the three -dimensional co-ordinates of that point. 
One might apply this program in a straightforward manner to a 
paint in one scene together successively with all points of the 
other. This would greatly reduce the possible "matches' 1 of 
features in the two scenes, but would, in general, not suffice to 
determine the three-dimensional locations of all the vertices. 
The problem of resolving the remaining ambiguities, as well as 
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developing a more efficient point -matching procedure seems to 

be quite complex. It seems likely that the organization that 
SEE imposes in che lines would provide a basis for using 
contextual clues to match the vertices. The problem is currently 
under investigation. 

A final problem worthy of mention is that of the 
determination of the stability of a configuration of prismatic 
solids, given a three-dimensional scene description of it. 
(Blum) The relevance of this problem derives from the fact 
that it is one of the goals of the object manipulator project to 
be able iu construct Structures from rectangular blocks. In 
order to direct, the object manipulation device* a program 
must first determine the order in which the blocks are to be 
set down, It may be possible 10 construct a sub -configuration 
of the desired configuration which has tht property that adding 
any additional block results in an unstable configuration. To 
forestall such a possibility, it is necessary to be able to 
detect instability in searching for an appropriate order in which 
to build the structure, 
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